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PREFACE 


This  paper  describes  work  done  under  USAF  Contract  No.  F49650-88-D5001,  Delivery 
Order  5032,  Evaluate  MPT  Resource  Requirements  During  Concept  Trade  Studies.  The  work 
was  performed  under  AL/HRM  Work  Unit  77191927  by  Systems  Research  and  Applications 
Corporation.  The  objective  of  the  effort  was  to  conduct  a  proof  of  concept  study  for  a  system  of 
models  proposed  to  estimate  the  life-cycle  cost  of  manpower,  personnel,  and  training  resources  of 
weapon  systems  proposed  during  Concept  Exploration. 

This  paper  presents: 

•  A  description  of  the  analyses  conducted  to  estimate  manpower  and  training  life- 
cycle  costs; 

•  A  comparison  of  the  life-cycle  costs  of  the  two  alternatives  proposed  in  the  proof 
of  concept  study;  and 

•  An  evaluation  of  the  viability  of  the  system  of  models  planned  for  development. 
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L  INTRODUCTION  AND  BACKGROUND 


1.1.  Goal  of  the  SYSMOD  Program 

This  paper  describes  the  activities  and  findings  of  the  second  task  of  the  SYSMOD 
(weapon  system  optimization  model)  research  program  being  conducted  by  the  Manpower  and 
Personnel  Division  of  the  Human  Resources  Directorate  of  the  Armstrong  Laboratory 
(AL/HRM).  The  goal  of  the  SYSMOD  program  is  to  develop  a  set  of  analysis  models  that  will 
allow  decision  makers  to  consider  the  manpower,  personnel,  and  training  (MPT)  resource 
availability  and  requirements  of  proposed  weapon  systems  when  choosing  among  alternatives 
during  the  very  early  phases  of  the  Weapon  System  Acquisition  Process  (WSAP).  SYSMOD 
concentrates  on  the  Pre-Concept  and  Concept  Exploration  phases  of  the  WSAP  whereas  a  second 
AL/HRM  research  thrust,  called  the  MPT  Decision  Support  System,  emphasizes  the  later  phases 
beginning  with  Demonstration  and  Validation. 

The  initial  version  of  SYSMOD  concentrates  on  aircraft  systems  but  it  will  consider  other 
systems  as  well  in  the  future.  The  scope  of  SYSMOD  is  currently  limited  to  enlisted  maintainers 
and  support  personnel.  In  practice,  SYSMOD  concentrates  on  enlisted  maintainers  and  direct 
support  personnel  on  the  flight  line  and  at  the  intermediate  level  (also  referred  to  as  shop  or  base 
level)  of  maintenance.  Base  operating  support  and  depot  maintenance  personnel  will  be  modeled 
later.  The  MPT  resources  modeled  include  number  of  positions  by  Air  Force  Specialty  (AFS)  and 
grade  or  skill  level,  and  training  requirements  for  people  filling  those  positions. 

In  Pre-Concept,  SYSMOD  estimates  the  ability  of  the  expected  force  structure  to  provide 
the  MPT  resources  to  maintain  a  new  or  modified  system.  In  Concept  Exploration,  SYSMOD 
models  the  maintenance  of  the  proposed  systems  to  determine  the  maintenance  hardware  and 
MPT  resources  required  to  provide  the  planned  system  availability,  where  system  availability  is  a 

measure  such  as  percent  time  ready  to  fly  or  number  of  sorties  per  day  the  system  can  fly.  Since 

\ 

MPT  resources  can  be  traded  off  with  hardware  resources  (number  of  spares  and  support 
equipment),  the  relative  costs  of  both  are  estimated.  Thus  SYSMOD  estimates  the  life-cycle 
costs  (LCCs)  of  both  MPT  and  hardware  resources.  Ideally,  SYSMOD  assists  the  analyst  in 
determining  for  each  proposed  system  the  combination  of  MPT  and  hardware  resources  that 
achieves  the  required  system  availability  at  a  minimum  LCC.  These  results  can  then  be  used  in 
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tradeoff  studies  to  determine  which  proposed  systems  should  be  advanced  to  Demonstration  and 
Validation. 

The  study  conducted  in  the  first  SYSMOD  task  proposed  a  conceptual  framework  for 
SYSMOD  and  produced  a  first  version  of  a  research  and  development  ^&D)  plan  (Rue,  1991). 
The  conceptual  fi'amework  is  divided  into  two  parts:  one  for  analysis  models  addressing  MPT 
resource  availability  issues  in  the  Pre-Concept  phase  of  the  acquisition  process  and  a  second  for 
analysis  models  addressing  MPT  resource  requirement  issues  in  the  Concept  Exploration  phase. 


1.2.  Goals  of  This  Task 

Because  the  development  of  SYSMOD  for  Concept  Exploration  offers  both  the  bigger 
challenge  and  the  greater  reward,  the  current  effort  focused  on  a  proof  of  concept  study  for 
SYSMOD  in  Concept  Exploration.  The  goals  of  this  second  SYSMOD  effort  were  to  determine: 

1.  whether  the  data  exist  to  support  the  proposed  system  of  models  and 

2.  whether  the  conceptual  framework  provides  a  sound  basis  for  estimating  the 
MPT  resource  requirements  and  LCCs  of  alternatives  proposed  during  Concept 
Exploration. 

In  the  proof  of  concept  study,  existing  models  designed  for  studies  conducted  later  in  the 
acquisition  process  were  used  to  provide  a  first  definition  of  data  requirements  for  the  framework. 

Existing  data  were  examined  to  determine  whether  they  satisfied  the  requirements  imposed. 
Problems  identified  during  the  proof  of  concept  study  provided  insight  into  the  soundness  of  the 
conceptual  fi'amework  and  helped  to  refine  the  R&D  plan  proposed  in  the  first  study. 


1.3.  Overview  of  SYSMOD  Conceptual  Framework  for  Concept  Exploration 

Concept  Exploration  is  a  period  when  a  number  of  weapon  systems  are  proposed  and 
evaluated  in  trade  studies  for  their  capacity  to  counter  the  threat  that  initiated  the  WSAP . 
SYSMOD  should  be  used  in  these  studies  to  compare  the  MPT  resources  required  by  the 
proposed  weapon  systems.  Thus  SYSMOD  allows  MPT  factors  to  influence  the  design  of  the 
weapon  system  by  including  MPT  resource  constraints  and  LCCs  in  the  trade  studies.  Ideally, 
SYSMOD  should  also  be  flexible  enough  to  handle  new  data  that  become  available  in  the  later 
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phases  of  the  WSAP  so  the  analyst  has  a  consistent  set  of  data  bases  and  tools  to  produce  updated 
MPT  resource  estimates. 

Since  SYSMOD  will  be  used  to  estimate  the  LCCs  of  the  maintenance  hardware  and  MPT 
resources  required  for  each  weapon  system  proposed  during  Concept  Exploration,  it  cannot 
require  elaborate  detail  or  long  periods  of  model  preparation  or  analysis.  In  fact,  detailed  data  are 
usually  not  available  for  proposed  weapon  systems.  SYSMOD  could  be  able  to  model  both 
peacetime  and  wartime  levels  of  performance.  The  level  of  performance  is  specified  in  terms  of 
flying  hours  per  aircraft  per  day  or  sorties  per  day  (where  sortie  lengths  are  known  or  can  be 
estimated).  Thus  the  maintenance  capacity  will  be  designed  to  ensure  aircraft  are  available  for  the 
required  flying  hours  or  sorties  per  day;  or  conversely,  that  aircraft  are  down  no  more  than  the 
remaining  hours  per  day. 

SYSMOD  will  be  used  to  estimate  the  MPT  LCCs  of  each  proposed  aircraft  so  they  can 
be  compared.  For  each  aircraft,  the  user  must  input  a  description  of  the  hardware  that  comprises 
the  aircraft  and  the  proposed  support  concept.  Figure  1  displays  a  list,  based  on  MIL-STD-780F, 
of  the  two-digit  work  unit  code  (WUC)  categories  used  to  represent  the  systems  that  comprise  an 
aircraft.  SYSMOD  models  an  aircraft  using  the  three-digit  WUC  categories,  called  sets.*  A  new 
aircraft  is  represented  as  a  hybrid  of  systems  and  sets  from  existing  aircraft.  The  system  and  set 
parameters  required  to  model  the  maintenance  of  an  aircraft  such  as  mean  time  between  failure 
(MTBF),  mean  time  to  repair  (MTTR),  and  maintenance  task  crew  size  are  drawn  from  the 
comparison  system  data  base  and  modified  by  the  user  for  any  changes  anticipated  for  the  new 
system.^  The  user  must  describe  the  support  concept  for  the  proposed  weapon  system.  The 
support  concept  description  includes  the  levels  of  repair  for  systems  and  sets,  the  assignment  of 
maintenance  tasks  to  AFSs,  and  the  shift  structure  (length  of  each  shift  and  the  number  of  airmen 
in  each  AFS  assigned  to  each  shift).  The  user  must  also  describe  the  weapon  system  performance 
requirements  in  terms  of  number  of  aircraft  simulated  and  required  flying  schedule  so  that 
SYSMOD  can  determine  whether  the  maintenance  resources  being  simulated  are  sufficient  to 
provide  the  required  aircraft  availability  rate. 


'  For  example,  WUC  63000  rq)resents  the  UHF  Communications  System.  WUC  63A00  is  the  UHF  Electronics 
Set  and  WUC  63B00  is  the  Integrated  Com-Nav  Control  Set . 

^  The  comparison  system  for  Concept  Ejq)loration  studies  may  be  simply  the  parent  ^stem. 
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wuc 

Description 

11 

Airframe 

12 

Crew  Station 

13 

Landing  Gear 

14 

Flight  Controls 

23 

Engine 

24 

Auxiliary  Power  Plant 

41 

Environmental  Control  System 

42 

Electric  Power  System 

44 

Lighting  System 

45 

Hydraulic/Pneumatic  Systems 

46 

Fuel  System 

47 

Oxygen  System 

49 

Miscellaneous  Utilities 

51 

Flight  Instruments 

55 

Malfunction  Analysis  Recorder 

62 

VHF  Communications  System 

63 

UHF  Communications  System 

64 

Interphone  System 

65 

Identification  Friend/Foe  (IFF) 

66 

Emergency  Radio  System 

71 

Radio  Navigation  System 

72 

Radar  Navigation  System 

74 

Fire  Control  System 

75 

Weapon  Delivery  System 

76 

Penetration  Aids  and  Electronic 
Countermeasures 

Figure  1.  Two-Digit  Work  Unit  Codes 


Figure  2  depicts  the  SYSMOD  architecture  proposed  in  the  first  study  (Rue,  1991).  The 
major  components  of  SYSMOD  are  a  comparison  systems  data  base  from  which  a  baseline 
comparison  system  (BCS)  can  be  formed,  a  simulation  model  of  the  maintenance  of  the  aircraft, 
an  LCC  model  for  MPT  resources  and  selected  hardware  resources,  and  tradeoff  models  to 
investigate  alternative  combinations  of  maintenance  hardware  and  MPT  resources. 
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Figure  2.  SYSMOD  During  Concept  Exploration 

The  comparison  systems  data  base  allows  the  SYSMOD  user  to  represent  the  maintenance 
characteristics  of  the  systems  and  sets  on  the  proposed  aircraft  by  selecting  the  most  similar 
current  system  from  the  data  base  and  modifying  data  describing  its  maintenance  to  bring  them  in 
line  with  the  proposed  aircraft. 

The  simulation  generates  maintenance  demand  as  a  function  of  the  flying  schedule  and  the 
maintenance  parameters  of  the  new  aircraft.  The  simulation  attempts  to  satisfy  the  maintenance 
demand  using  the  maintenance  resources  provided  through  the  parameters  of  the  support  concept 
such  as  the  assignment  of  tasks  to  AFSs  and  the  shift  structure.  The  simulation  "plays  out"  the 
flying  and  maintaining  of  aircraft  for  an  input  period  of  time  and  reports  the  results.  If  the  aircraft 
do  not  meet  the  flying  schedule,  the  user  should  change  the  parameters  of  the  support  concept  in 
order  to  meet  the  schedule.  This  may  be  an  iterative  process. 

Once  the  simulation  indicates  that  the  new  aircraft  can  meet  the  performance  requirements 
(represented  by  the  flying  schedule),  SYSMOD  estimates  the  LCCs  of  maintaining  the  new 
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aircraft.  The  LCC  model  computes  costs  of  manpower  positions  at  both  the  flightline  and 
intermediate  levels  of  maintenance  and  computes  the  cost  of  selecting  and  training  people  to  fill 
those  positions.^  The  LCC  model  also  computes  costs  of  selected  hardware  resources  provided 
for  maintenance  including  support  equipment  and  spares. 

The  tradeoff  component  of  SYSMOD  can  be  viewed  as  a  collection  of  models  that 
evaluate  various  combinations  of  maintenance  resources  searching  for  one  that  meets  system 
performance  requirements  at  the  lowest  LCC.  Once  the  tradeoff  models  identify  resource  trades, 
the  simulation  model  can  be  updated  to  reflect  the  changes  and  be  rerun  to  reevaluate  system 
performance  (sortie  rate  or  flying  hours)  and  LCC.  The  tradeoff  process  continues  until  the  user 
is  satisfied  the  LCC  of  the  maintenance  resources  required  by  the  system  cannot  be  reduced 
further. 


SYSMOD  produces  measures  of  maintenance  activity  from  the  simulation  including 
utilization  rate  of  each  AFS,  down  time  for  aircraft,  and  numbers  of  maintenance  actions.  The 
achieved  sortie  rate  will  also  be  output  and  compared  with  the  required  rate.  Several  categories 
of  LCCs  will  be  estimated  including  manpower  costs  by  AFS  and  skill  level  or  grade,  selection 
and  training  costs  by  AFS,  and  costs  of  spares  and  support  equipment  involved  in  the  MPT  cost 
comparisons. 


1.4.  Overview  of  the  Proof  of  Concept  Study 

The  system  of  data  bases  and  models  described  above  was  the  subject  of  the  proof  of 
concept  study.  Because  the  main  goals  of  the  proof  of  concept  study  were  to  check  the 
availability  of  data  and  the  viability  of  the  conceptual  framework  for  SYSMOD  in  Concept 
Exploration,  the  study  concentra.ted  on  MPT  issues  and  not  hardware  issues.  Thus  the  differences 
in  the  alternatives  studied  were  primarily  in  the  MPT  resources  they  required  causing  the  study  to 
concentrate  on  MPT  resource  data  and  models.  Several  other  factors  helped  define  the  specific 
issues  addressed.  To  ensure  that  data  fi-om  the  Concept  Exploration  phase  still  existed,  the  study 
focused  on  an  aircraft  still  in  the  acquisition  process.  In  order  to  attract  broader  interest  in  the 
study  than  from  just  those  interested  in  SYSMOD,  the  differences  between  the  alternatives 
studied  hinged  on  a  topic  of  current  concern  to  the  Air  Force.  To  limit  the  data  gathering  and 

^  The  LCC  model  can  also  compute  the  cost  of  additional  base  operating  support  personnel  needed  as  a  result  of 
maintenance  manpower  required  for  the  new  aircraft. 
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analysis  eflfort,  the  study  concentrated  on  a  single  flightline  maintenance  AFS  and  the  systems  and 
sets  the  AFS  maintains. 

The  aircraft  studied  was  the  Advanced  Tactical  Fighter  or  ATF.  The  study  focused  on  the 
ATF's  Integrated  Communications-Navigation-Identification  Avionics  (ICNIA)  systems  and  the 
AFS  that  will  maintain  them  on  the  flightline,  hereafter  referred  to  as  Air  Force  Specialty  Code 
(AFSC)  452XXC  (ATF  ICNIA  Systems).'*  The  differences  between  the  two  alternatives  studied 
emanated  from  a  difference  in  support  concept.  In  the  first  alternative,  the  ATF  squadrons  were 
assigned  to  bases  as  independent  units  with  their  own  maintenance  squadrons  providing  flightline 
maintenance  of  the  aircraft.  In  the  second  alternative,  the  ATF  squadrons  were  assigned  to 
composite  units  at  bases  with  F-15  aircraft.  The  AFSC  that  conducts  flightline  maintenance  on 
similar  systems  for  the  F-15  is  452X1C  (F-15  Communication,  Navigation,  and  Penetration  Aids 
(Comm/Nav/Pen  Aids)  Systems).*  In  the  second  alternative,  flightline  maintenance  of  the  ATF 
was  provided  by  a  composite  maintenance  squadron  with  some  maintainers  primarily  assigned  to 
the  ATF  and  others  primarily  assigned  to  the  F-15.  However,  maintainers  in  the  composite  units 
were  cross  trained  to  the  5-level  in  some  maintenance  tasks  for  the  other  aircraft.^  Because  the 
Air  Force  is  currently  forming  composite  units  at  several  bases,  the  results  of  this  study  should 
also  be  of  interest  to  analysts  who  are  studying  the  implementation  of  these  units. 

In  the  first  alternative,  the  Logistics  Composite  Model  (LCOM)  was  used  to  simulate  the 
maintenance  for  a  squadron  of  24  ATF  aircraft  to  estimate  the  number  of  flightline  maintainers 
needed  for  the  ICNIA  systems.  The  same  model  also  simulated  maintenance  for  a  squadron  of  24 
F-15  aircraft  to  estimate  the  flightline  maintenance  manpower  needed.  Although  both  aircraft 
were  run  in  the  same  model,  the  model  treated  the  maintenance  of  the  aircraft  independently. 
ATF  resources  could  only  be  used  on  the  ATF,  and  F-15  resources  could  only  be  used  on  the  F- 
15. 


In  the  second  alternative,  both  squadrons  of  aircraft  were  again  represented  in  the  same 
LCOM  run.  For  this  alternative,  the  study  took  advantage  of  an  LCOM  capability  that  allows  the 


'*  The  systems  in  ICNIA  include  WUCs  62-66, 71, 72,  and  74. 

*  Systems  included  in  Comm/Nav/Pen  Aids  include  WUCs  63, 65, 71,  and  76. 

^  Because  they  are  only  qualified  at  the  5-level  in  some  of  the  tasks,  the  cross  trainees  were  not  granted  a  5-level 
in  the  secondary  AFS. 
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user  to  identify  an  alternate  or  secondary  resource  for  performing  maintenance  when  the  primary 
resource  is  unavmlable.  In  the  second  alternative,  a  number  of  ATF ICNIA  flightline  maintenance 
tasks  had  F-15  maintainers  identified  as  a  secondary  resource  for  performing  the  tasks.  Similarly, 
a  number  of  F-15  Comm/Nav/Pen  Aids  flightline  maintenance  tasks  had  ATF  maintainers 
identified  as  a  secondary  resource  for  accomplishing  the  tasks.  Members  of  the  secondary  AFS 
were  assumed  to  take  longer  to  perform  tasks  on  their  secondary  aircraft  than  airmen  from  the 
primary  AFS. 

Assuming  the  ATF  and  F-15  aircraft  could  meet  their  flying  schedules  in  both  alternatives, 
the  choice  between  the  two  rested  on  a  comparison  of  their  LCCs.  The  first  alternative  provided 
a  baseline  for  LCCs.  In  the  second,  savings  from  the  baseline  might  result  from  a  reduction  in 
required  manpower  due  to  the  availability  of  a  secondary  resource.  Also,  costs  above  the  baseline 
might  result  from  cross  training  both  ATF  and  F-15  maintainers  for  maintenance  tasks  on  the 
other  aircraft.  The  net  effect  of  the  savings  and  extra  costs  determined  whether  the  second 
alternative  was  less  expensive  than  the  first.  The  study  assumed  that  the  Air  Force  would  form 
one  unit  of  F-15  and  ATF  squadrons  per  year  for  nine  years,  then  keep  those  nine  units  until  year 
21  when  it  would  reduce  the  number  of  units  by  one  to  eight.  The  reduction  of  one  unit  per  year 
was  assumed  to  continue  until  year  28  when  the  remaining  two  units  would  be  deactivated. 

The  next  section  of  the  report  describes  the  data  gathered  and  the  analyses  conducted  to 
compare  the  two  alternatives.  The  section  also  details  problems  encountered  with  the  data  and  in 
conducting  the  analyses. 
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2.  DATA  DEVELOPMENT 


2.1.  Scope 


The  proof  of  concept  study  was 
aimed  at  the  primary  processes  required  in 
SYSMOD;  developing  input  data  based  on  a 
BCS  data  base,  simulating  the  maintenance 
of  the  aircraft,  evaluating  performance,  and 
estimating  LCCs.  It  did  not  exercise  two  of 
the  proposed  components  of  SYSMOD  as 
portrayed  in  Figure  2.  First,  the  study  did 
not  conduct  any  tradeoff  studies  aimed  at 
reducing  the  LCC  of  each  of  the  alternatives. 

Tradeoff  studies  might  have  examined  the 
interaction  of  the  MTBF  or  MTTR  of  the 
systems  modeled  with  the  MPT  resources 
required  for  maintenance.  However, 
maintenance  parameters  of  the  hardware 
were  considered  fixed  for  the  study. 
Tradeoff  studies  might  also  have  considered 
the  interaction  among  MPT  resources  as  a 
result  of  a  different  assignment  of  tasks  to 
AFSs  or  of  using  new  selection  criteria  for 
entry  to  an  AFS.’  The  second  component  of 
SYSMOD  that  was  not  exercised  in  this 
study  was  the  estimation  of  hardware  LCCs. 

Because  hardware  resources  were  fixed  for 
the  study,  hardware  costs  were  not  needed 
to  compare  the  two  alternatives. 


wuc 

Descripticai 

62000 

VHF 

62100 

VHF  Radio  Set  (AN/ARC182) 

62X00 

Associated  VHF  Equipment  Set 

mm 

63200 

UHF  Radio  Set  (AN/ARC159) 

63500 

Digital  Data  Communications  Set 

63Y00 

Associated  UHF  Equipment  Set 

.  64000 

64X00 

Associated  Interphone  Equipment  Set 

65000 

Idemificsjto  Fxietjd  or 

65300 

Transponder  Set  (AN/APXlOO) 

65Y00 

Associated  IFF  Equipment  Set 

66000 

Emergency  Radio  Syatos 

66100 

Radio  Beacon  Set 

7JOOO 

Ra<3goKaviga6on  System 

71300 

Tactical  Air  Navigation  (TACAN)  Set 

71D00 

Receiving  Decoding  Set 

71Y00 

Associated  Radio  Navigation  Equipment  Set 

72000 

RaderKavigation  Systei 

72200 

Electronic  Altimeter 

72900 

Radar  Beacon  Set 

72R00 

Radar  Navigation  Related  Equipment 

72Y00 

Radar  Navigation  Associated  Equipment  , 

74000 

Weapons  Control  Syi^ems 

Figures.  BCSforATF 


^  The  two  alternatives  studied  are  essentially  task  assignment  alternatives  where  the  AFSs  considered  are  from 
two  different  aircraft. 
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2.2.  Input 


2.2.1.  BCS  and  LCOM  Data 

BCS  data  for  the  ATF  were  taken  from  sources  developed  in  the  mid-1980's  by  AL/HRG 
in  conjunction  with  their  development  of  MPT  analysis  methodologies  and  software  under  the 
Small  Unit  Maintenance  Manpower  Analyses  (SUMMA)  project.  Data  remaining  from  that  effort 
include  LCOM  data  required  to  model  three  levels  of  maintenance  for  the  ICNIA  systems, 
although  the  data  only  portray  depot  maintenance  as  delays  for  repair  of  equipment  that  is  not 
repairable  at  the  intermediate  level  (Not  Repairable  This  Station  (NRTS)).  Figure  3  lists  the 
systems  and  sets  in  the  BCS  for  ATF  ICNIA  systems;  all  are  from  the  Navy's  F-18.  Appendix  A 
lists  the  maintenance  action  code  for  each  maintenance  task  performed  on  ATF  ICNIA  systems, 
crew  size  required  for  the  task,  and  the  expected  task  performance  time,  all  of  which  were 
extracted  from  the  LCOM  data  set. 

LCOM  data  for  the  F- 15  were  taken 
from  an  Air  Combat  Command  (ACC) 

LCOM  model  used  by  AL/HRG  in  the  late 
1980s.  The  F-15  data  also  model  three  levels 
of  maintenance  and  limit  the  representation  of 
depot  actions  to  delays  for  hardware  that  is 
NRTS.  Figure  4  lists  the  Comm/Nav/Pen 
Aids  systems  and  sets  in  the  F-15  LCOM 
model.  Appendix  B  also  displays  the 
maintenance  action  code  for  each 
maintenance  task  performed  on  F-15 
Comm/Nav/Pen  Aids  systems,  crew  size 
needed,  and  expected  task  performance  time 
as  given  in  the  LCOM  data  set. 


2.2.2.  Support  Concept 

As  indicated  in  the  discussion  above, 
the  support  concept  for  the  ATF  and  the  F- 
15  included  three  levels  of  maintenance  for  Figure  4.  BCS  for  F-15 

both  of  the  alternatives  although  depot 


wuc 

DescripfEon. 

63O00 

UHF  Cominimicatitms  System 

63A00 

UHF  Communications  Set 

63B00 

Integrated  Comm/Nav/IFF  Set 

65mo 

l&kntification.  Friend  or  Foe  System 

65A00 

Transponder  Set 

65B00 

IFF  InteiTogator  Set 

71O0O 

Radio  FJavigation  System. 

71C00 

Instrument  Landing  Set  (ILS) 

71D00 

Tactical  Air  Navigation  (TACAN)  Set 

71F00 

Attitude  Heading  Reference  Set  (AHRS) 

lizm 

AN/ARN-l  18  TACAN  Insfl 

76000 

Tactical  Electonic  Warfere  System 
(TEWS) 

76A00 

Counteimeasures  Receiving  Set  (AN/ALR- 
56) 

76C00 

Interference  Blanker 

76G00 

Coimtermeasures  Set  (AN/ALQ-128) 

76H00 

Countermeasures  Set  (AN/ALQ-135V) 
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manpower  was  not  modeled.  The  availability  of  spares  and  support  equipment  were  also  the  same 
for  both  alternatives.  The  only  aspect  of  the  support  concept  that  varied  was  under  the  second 
alternative,  the  composite  unit  alternative,  where  a  secondary  AFS  was  available  for  performing 
some  maintenance  actions.  The  rest  of  this  section  describes  the  process  used  to  modify  the  BCS 
LCOM  data  to  model  the  availability  of  a  secondary  AFS. 

Choosing  Tasks  for  Performance  by  Secondary  AFS:  The  first  step  in  preparing 
LCOM  data  was  to  select  tasks  that  members  of  the  secondary  AFS  would  perform  and  for  which 
they  should  be  cross  trained.  The  study  assumed  that  airmen  in  the  composite  unit  maintenance 
organization  would  be  primarily  assigned  to  one  aircraft,  but  that  they  could  be  cross  utilized  on 
the  other  aircraft  depending  on  relative  workload  and  mission  priority  for  the  two  types  of 
aircraft.  Although  some  people  interviewed  during  the  study  felt  that  the  Air  Force  would  only 
cross  train  the  entire  set  of  tasks  for  an  AFS,  as  is  done  during  retraining  from  one  AFS  to 
another,  the  study  took  a  less  restrictive  view  of  the  choice  of  tasks  and  considered  cross  training 
subsets  of  an  AFS's  tasks.  The  process  of  considering  task  subsets  for  cross  training  forced  the 
study  to  investigate  whether  data  exist  to  model  training  requirements  and  costs  at  more  detailed 
levels  than  the  total  of  all  tasks  assigned  to  an  AFS. 

Three  main  factors  determined  the  selection  of  tasks  to  be  cross  trained.  First,  cross 
training  costs  were  to  be  minimized.  Thus,  if  two  groups  of  tasks  produced  the  same  manpower 
savings,  the  one  with  the  smaller  training  cost  should  be  chosen  to  minimize  overall  costs.  In 
general,  choosing  tasks  that  require  similar  sets  of  knowledge,  skills,  and  abilities  (KS  As)  would 
tend  to  limit  training  costs. 

The  second  factor  affecting  task  selection  was  safety.  Because  the  Air  Force  is  a 
conservative,  safety  conscious  organization,  the  study  assumed  that  the  Air  Force  would  authorize 
members  of  the  primary  AFS  only  to  certify  completion  of  maintenance  tasks.  As  a  result,  the 
study  required  one  person  from  the  primary  AFS  on  each  crew  and  did  not  consider  assigning 
one-person  tasks  to  the  secondary  AFS. 

A  third  factor  affecting  cross  training  task  selection  was  the  reduction  of  manning 
requirements  for  the  primary  AFS.  The  flexibility  afforded  by  having  a  secondary  AFS  should 
produce  manpower  economies  for  the  primary  AFS  1)  by  permitting  a  limited  number  of  primary 
AFS  personnel,  in  conjunction  with  secondary  AFS  personnel,  to  perform  more  primary  aircraft 
maintenance,  2)  by  allowing  primary  AFS  personnel  to  concentrate  on  more  specialized  tasks. 
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thus  completing  the  same  total  maintenance  more  efficiently  and  so  faster,  and  3)  through 
efficiencies  as  in  2),  by  allowing  fewer  people  from  the  primary  AFS  to  complete  the  total 
maintenance  effort  in  the  same  time. 

In  light  of  the  factors  above,  the  first  step  in  selecting  tasks  for  cross  training  was  to 
identify  groups  of  tasks  requiring  similar  KSAs  in  order  to  restrict  the  search  to  combinations  with 
limited  training  costs.  The  study  examined  two  methods  for  identifying  these  groups  of  tasks,  the 
co-performance  method  and  the  WUC  method. 

The  first  method  for  grouping  tasks  is  the  co-performance  method.  Task  co-performance 
is  a  measure  of  the  tendency  for  tasks  to  be  performed  by  the  same  people;  it  is  reported  as  a 
percentage.  Two  tasks  have  high  co-performance  if  most  of  the  people  who  perform  one  task 
also  perform  the  other.  The  basis  for  using  this  measure  in  the  study  was  the  assumption  that  the 
need  for  efficiency  in  the  workplace  causes  supervisors  to  repeatedly  assign  airmen  to  tasks 
requiring  a  relatively  narrow  set  of  KSAs.  As  a  result,  the  supervisor  ensures  the  workload  facing 
the  organization  is  accomplished,  and  the  airmen  become  proficient  at  the  tasks  and  tend  to 
specialize  within  their  AFSs. 

To  examine  co-performance,  the  study  needed  to  correlate  the  LCOM  tasks  which  are 
based  on  Maintenance  Data  Collection  System  (MDCS)  data  gathered  for  the  maintenance  and 
logistics  communities  with  Occupational  Survey  (OS)  data  used  by  the  training  community  and 
serving  as  the  basis  for  co-performance  analyses.  Several  recent  reports  describe  research 
conducted  by  both  AL/HRT  and  AL/HRM  into  streamlining  or  automating  the  process  of 
mapping  MDCS  data  to  OS  data  (Wagner,  1986  and  Metrica,  1991).  This  study's  mapping 
procedure  borrowed  heavily  from  Metrica's  findings,  although  it  did  not  use  Metrica's  automated 
mapping  procedure  due  to  lack  of  data.  Appendix  C  describes  the  process  followed  for  mapping 
LCOM  tasks  to  OS  tasks  and  displays  the  results  for  the  F- 15.* 

Co-performance  measures  were  produced  by  applying  Comprehensive  Occupational  Data 
Analysis  Programs  (CODAP)  routines  to  OS  data  for  tasks  identified  in  the  LCOM  mapping,  for 
members  of  AFS  452X1 C  with  four  years  of  service  or  less.  The  study  assumed  that  supervisors 
will  treat  cross  trainees  as  relatively  inexperienced  maintainers  and  will  assign  them  to  tasks 


*  Because  the  BCS  for  the  ICNIA  systems  on  the  ATF  was  the  Navy's  F-18  and  hence  Air  Force  OS  data  did  not 
exist,  the  study  did  not  use  the  co-performance  method  for  the  ATF. 
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normally  assigned  to  airmen  with  four  years  of  service  or  less.  Appendix  D  describes  the  study's 
process  of  using  co-performance  measures  to  cluster  tasks  into  co-performance  groups  in  order  to 
identify  groups  of  tasks  as  candidates  for  cross  training. 

Figure  5 

summarizes  the  results 
of  the  clustering.  In 
Figure  5  the  height  of 
each  rectangle  indicates 
the  range  of  co¬ 
performance  measures 
for  tasks  in  the  cluster 
and  the  width  of  the 
rectangle  is 

proportional  to  the 
number  of  tasks  in  the 
cluster.  The  lines 

connecting  rectangles 
indicate  how  the  5.  F-15  Tasks  Aggregated  by  Co-Performance 

clusters  combine  to 

form  larger  groups.  Most  of  the  hands-on  maintenance  tasks  were  highly  co-performed  with  one 
another  so  there  were  few  prominent  breaks  between  clusters  of  tasks.  However,  the  clusters 
tended  to  contain  all  tasks  associated  with  one  or  more  three-digit  WUCs.  Figure  5  lists  the 
WUCs  that  comprise  each  cluster.  This  result  gave  rise  to  the  second  approach,  the  WUC 
approach,  to  identifying  sets  of  tasks  for  cross  training. 

The  WUC  approach  to  identifying  tasks  with  similar  KSAs  forms  a  cluster  of  tasks  for 
each  three-digit  WUC  set.  Thus  maintenance  tasks  associated  with  a  three-digit  WUC  form  the 
smallest  clusters  of  tasks.  The  hypothesis  behind  this  approach  is  that  limiting  the  types  of 
equipment  maintained  yields  a  narrow  range  of  KSAs  for  cross  training.  Because  the  approach 
does  not  use  OS  data,  it  can  be  applied  to  the  ATF  (F-18)  as  well  as  to  the  F-15.  Larger  clusters 
of  tasks  can  be  formed  by  grouping  clusters  within  the  same  two-digit  WUC  system. 

Both  the  co-performance  method  and  the  WUC  method  seek  groups  of  tasks  with  a 
narrow  range  of  KSAs  and  so  should  tend  to  identify  similar  groups  of  tasks.  Since  the  co- 
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performance  method  is  less  restrictive,  it  may  yield  groups  that  are  composed  of  three-digit  WUC 
sets  from  different  systems.  For  small  clusters,  it  may  even  group  on  maintenance  action  (e.g. 
cluster  troubleshooting  tasks  together).  The  co-performance  method  also  tends  to  isolate  difficult 
or  complex  tasks,  tasks  performed  by  only  a  few,  specialized  5  or  7-levels,  into  small  clusters. 
Tasks  such  as  these  were  not  considered  for  cross-training  in  this  study  because  they  would  not  be 
assigned  to  inexperienced  maintainers. 

On  the  basis  of  the  above  finding  regarding  complex  tasks,  the  WUC  approach  was 
modified  to  identify  and  eliminate  tasks  requiring  specialization.  A  recent  study  proposed  a 
simple  process  for  identifying  these  tasks  when  OS  data  are  available  (Driskill,  1987).  The 
process  identifies  tasks  requiring  specialization  by  selecting  those  with  a  learning  difficulty  greater 
than  one  standard  deviation  above  the  mean,  and  with  the  percent  members  performing  less  than 
one  standard  deviation  above  the  mean.^  The  study  applied  this  additional  process  to  the  F-15 
tasks  but  found  that  none  were  eliminated.*** 


Because  the  WUC  method  could  be  applied  to  both  the  ATF  and  the  F-15,  the  study 
chose  to  start  with  the  3-digit  WUC  clusters  of  tasks  resulting  from  that  method.  The  second  step 
in  selecting  tasks  was  to  eliminate  any  one-person  tasks  from  clusters.  The  study  eliminated  these 
tasks  because  of  their  potential  for  safety 
problems  if  they  were  performed  by  members  of 
the  secondary  AFS  acting  alone. 

The  modified  groups  of  tasks  were  ranked 
by  their  contribution  to  the  expected  workload  for 
the  primary  AFS.  The  expected  workload  for  a 
task  was  computed  as  the  product  of  the  number 
of  sorties  per  day  for  the  squadron,  the  per  sortie 
probability  of  the  task  being  required,  the 
expected  performance  time  of  the  task,  and  the 
number  in  the  crew.  The  ranking  for  each  set  of 

tasks  was  determined  by  summing  the  expected  Figure  6.  Expected  Workload  for  72 

ATF  Sorties 


WUC 

Expected  Workload 

62 

10.53 

65 

5.21 

71 

3.37 

72 

3.27 

66 

0.75 

74 

0.73 

63 

0.32 

64 

0.05 

®  The  tasks  selected  by  this  process  are  both  difScult  to  learn  and  are  performed  by  relatively  few  people. 
No  isolated  groups  were  identified  by  the  co-performance  method  either. 
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workloads  of  the  tasks  in  the  three-digit  WUC  cluster.” 

To  limit  later  data  gathering  and  analysis  efforts,  the  study  aggregated  three-digit  WUC 
clusters  into  two-digit  WUC  clusters.  Figure  6  displays  the  rank  ordering  of  two-digit  WUC 
clusters  for  the  ATF  and  Figure  7  displays  the  rank  ordering  for  the  F-15. 

The  third  step  was  to  choose  the  groups  of  tasks  from  the  rank  ordered  lists  to  be  cross 
trained.  Even  though  WUC  76  (Tactical  Electronic  Warfare  System)  tasks  were  ranked  at  the  top 
of  the  F-15  expected  workload  list,  they  were  eliminated  from  consideration  for  cross  training 
because  the  ATF  had  no  similar  group  of  tasks.  Once  the  WUC  76  group  was  dropped,  only  the 
top  two  groups  were  chosen  for  each  aircraft  in  this  study  in  order  to  limit  the  analyses  required 
for  LCC  estimation.  Thus  WUC  62  and  65  tasks  were  chosen  for  the  ATF  and  WUC  65  and  63 
tasks  were  chosen  for  the  F-15. 


Estimating  Task  Performance  Times 
for  the  Secondary  AFS:  Once  the  study 
identified  the  groups  of  tasks  chosen  for  cross 
training,  it  had  to  estimate  the  task  performance 
times  for  maintenance  crews  that  included  people 
from  both  the  primary  and  the  secondary  AFSs. 

Gathering  empirical  data  to  make  these  estimates 
would  be  difficult  for  both  technical  and  cost 
reasons.*^  The  study  sought  to  use  existing  data 
to  make  the  estimates.  The  most  promising  data 
came  from  studies  done  in  the  niid-1980s  by  Figure  7.  Expected  Workload  for  72  F- 
AL/HRG  as  a  part  of  their  SUMMA  research  Sorties 

program.  The  SUMMA  effort  gathered  task  performance  time  data  for  the  F-16  (Boyle,  1990) 
and  for  the  F-18  (no  report  published). 


WUC 

Expected  Workload 

76 

79.30 

65 

22.98 

63 

13.68 

71 

4.12 

”  Appendices  A  and  B  list  the  rank  ordered  sets  of  tasks  for  each  aircraft. 

”  Collecting  empirical  data  for  this  study  requires  crews  composed  of  both  Air  Force  and  Navy  persoimel  to  be 
cross  trained  and  then  timed  on  the  chosen  maintenance  tasks. 


15 


Because  the  SUMMA  data  for  the  F-18  included  many  of  the  LCOM  tasks  associated 
with  the  ICNIA  systems,  the  study  examined  the  F-18  data  first,  looking  for  a  simple  approach  to 
estimating  performance  times  for  mixed  crews.  The  SUMMA  effort  interviewed  maintainers  of 
the  F-18  to  collect  data  on  each  task  from  each  person  that  included  the  following: 

•  estimate  of  performance  time  for  a  crew  composed  of  people  from  a  secondary  AFS 

•  estimate  of  difficulty  for  the  task  (5-point  scale) 

•  estimate  of  electronic  knowledge  required  (7-point  scale) 

•  estimate  of  mechanical  knowledge  required  (7-point  scale) 

The  study  examined  these  data  looking  for  a  simple  method  for  estimating  the  performance  time 
for  a  crew  composed  of  people  fi’om  the  secondary  AFS.  The  study  assumed  that  airmen  fi'om 
the  secondary  AFS  would  not  perform  the  tasks  as  frequently  as  airmen  from  the  primary  AFS; 
thus  the  study  attempted  to  use  secondary  AFS  SUMMA  task  times.  Other  AL/HRM  research  on 
productive  capacity  (Leighton,  1992)  may  also  produce  a  method  to  estimate  task  performance 
times.  The  study  then  extended  the  method  to  estimate  the  performance  time  for  a  mixed  crew. 

The  results  of  the  examination  of  F-18  data  were  disappointing  (see  Appendix  E).  Only  a 
handful  of  subjects  were  interviewed,  and  their  responses  were  too  varied  to  yield  useful 
regression  models  for  estimating  times  for  secondary  crews.  The  study  then  turned  to  the 
SUMMA  data  for  the  F-16. 

The  SUMMA  data  for  the  F-16  included  estimates  from  20  to  24  people  for  each  of  the 
452X2C  tasks  studied  and  were  more  consistent  than  those  for  the  F-18.*^  The  study  examined 
functions  that  estimated  secondary  AFS  performance  time  as  a  function  of  primary  AFS 
performance  time,  the  SUMMA  estimates  of  task  difficulty  and  knowledge  required,  and  OS 
estimates  of  task  learning  difficulty.  The  best  regression  model  estimated  secondary  AFS 
performance  time  simply  as  a  function  of  primary  AFS  performance  time  where  secondary  time 
was  1.24  times  primary  time.  The  study  assumed  that  this  function,  based  on  F-16  data,  applied 
to  both  the  ATF  (F-18)  and  the  F-15.  It  applied  the  1.24  factor  to  primary  AFS  times  from 
LCOM  data  to  estimate  secondary  times  for  tasks  selected  for  cross  training. 


At  the  time  the  SUMMA  study  was  conducted  the  AFSC  for  F-16  Avionics  was  326X8C. 
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The  study  found  no  data  on  which  to  base  a  method  for  combining  the  primary  crew  and 
secondary  crew  times  into  a  mixed  crew  time.  The  study  assumed  that  the  mixed  time  was 
between  the  two  estimates  and  calculated  it  as  the  average  of  the  two  times.  Because  the  AFS 
crew  time  for  the  secondary  crew  was  1.24  times  the  primary  AFS  crew  time,  the  mixed  crew 
time  was  1.12  times  the  primary  crew  time.^'*  Note  that  all  of  the  tasks  selected  for  cross  training 
were  two-person  tasks  except  for  one  three-person  task.  Input  to  the  LCOM  model  identified 
mixed  crews  as  the  second  choice  for  completing  each  cross-trained  maintenance  task.  Options 
were  set  so  LCOM  first  searched  for  an  available  primary  crew  before  selecting  a  mixed  crew  to 
complete  a  task. 


2.2.3.  Operating  Concept  and  Performance  Requirements 

The  main  effect  of  the  operating  concept  on  the  study  was  in  the  generation  of 
maintenance  workload.  Because  only  a  few  tasks  were  chosen  for  cross  training,  the  availability 
of  a  secondary  AFS  would  not  have  much  of  an  effect  on  manpower  requirements  unless  the 
workload  for  the  AFS  was  large.  Thus  the  operating  concept  was  designed  to  call  for  an  intense 
flying  schedule  that  required  three  sorties  per  day  for  all  24  aircraft  of  each  type,  or  72  sorties  per 
day  for  each  squadron. 

The  performance  measures  for  the  system  were  given  in  terms  of  achieved  sorties  per  day 
versus  required  sorties.  The  study  attempted  to  provide  the  maintenance  units  in  the  two 
alternatives  with  just  enough  people  to  keep  manpower  from  becoming  a  limiting  factor  in 
meeting  the  required  sortie  rate,  thus  minimizing  the  manpower  costs  for  each  alternative. 


LCOM  requires  a  probability  distribution  as  input  for  task  performance  times.  The  LCOM  models  for  the 
ATF  and  F-15  use  normal  and  log-normal  distributions.  In  the  study,  the  mean  of  the  distribution  for  the  primary 
crew  was  multiplied  by  1.12  to  estimate  the  mean  for  the  distribution  for  the  mixed  crew.  The  variance  of  the 
distribution  for  the  mixed  crew  was  unchanged  from  that  for  the  primary  crew. 

As  shown  in  Section  3.1.2,  the  LCOM  model  for  the  F-15  could  not  achieve  three  sorties  per  day  per 
aircraft  even  with  unlimited  manpower. 
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3.  ESTIMATING  MANPOWER  AND  TRAINING  REQUIREMENTS 


3.1.  Manpower  Requirements 

The  study  used  LCOM  to  simulate  the  maintenance  of  aircraft  for  both  the  independent 
and  the  composite  maintenance  alternatives.  As  mentioned  in  Section  2.2.1,  existing  LCOM 
models  of  the  ATF  and  F- 15  were  used  in  the  study.  Note  that  the  manpower  requirements 
estimation  model  used  in  SYSMOD  does  not  need  to  be  as  detailed  as  the  LCOM  models  used  in 
the  study  and  should  be  designed  to  be  much  easier  to  run  than  the  LCOM  models  were. 
However,  the  LCOM  models  were  readily  available  and  allowed  the  study  to  concentrate  on  the 
objectives  of  the  task:  investigating  data  availability  and  evaluating  the  soundness  of  the 
conceptual  framework  for  SYSMOD. 


3.1.1.  Models  of  the  Two  Alternatives 

LCOM  is  a  Monte  Carlo  simulation  model  written  in  Simscript  11.5  that  represents  the 
generation  of  aircraft  sorties  as  events  that  are  scheduled  according  to  a  user  input  mission  plan. 
Both  scheduled  and  unscheduled  maintenance  are  modeled  as  task  networks  where  the  network 
branches  taken  depend  on  the  outcomes  of  random  draws.  For  instance,  a  troubleshooting  task 
might  be  followed  by  a  remove  and  replace  task  or  by  a  minor  repair  task,  depending  on  the 
random  draw.  The  tasks  in  the  networks  for  AFSs  452XXC  (ATF  ICNIA  Systems)  and  452X1 C 
(F-15  Comm/Nav/Pen  Aids)  are  listed  in  Appendices  A  and  B.  The  model  enters  the  networks 
whenever  maintenance  is  scheduled  or  an  aircraft  system  fails.  Aircraft  system  failures  occur 
according  to  probability  distributions  input  by  the  user.  Typically,  these  distributions  are  based  on 
studies  of  actual  failures  over  some  period  of  aircraft  operation  or  are  estimates  of  distributions 
for  systems  yet  to  be  built.  When  the  model  enters  a  maintenance  network,  it  assigns  a 
maintenance  crew  to  the  first  task  in  the  network  and  draws  a  random  task  time  from  the 
distribution  for  the  task  performance  time.  At  the  end  of  the  task,  LCOM  releases  the  crew  and 
proceeds  to  the  next  task  according  to  the  network  branches  and  any  random  draws  needed.  If 
LCOM  cannot  find  a  maintenance  crew,  it.  places  the  task  in  a  queue  (backorders  the  task)  until  a 
crew  is  available.  When  the  model  reaches  the  point  in  the  network  where  the  aircraft  is  repaired, 
it  releases  the  aircraft  to  fly  although  it  may  continue  through  the  network.  For  instance,  the 
network  may  continue  to  represent  the  repair  process  for  a  defective  line  replaceable  unit  (LRU) 
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that  is  sent  to  the  intermediate  maintenance  level  for  repair.  Because  LCOM  results  depend  on 
the  outcome  of  a  number  of  random  draws,  LCOM  models  are  usually  run  several  times  to 
develop  distributions  of  results. 

In  the  independent  maintenance  alternative,  LCOM  searched  only  the  primary  AFS  for  a 
maintenance  crew.  Those  airmen  were  trained  for  just  their  primary  aircraft  and  could  not  be 
used  on  the  secondary  aircraft.  In  the  composite  unit  alternative,  LCOM  searched  for  a  work 
crew  in  the  primary  AFS  first.  But  for  those  tasks  that  were  cross  trained,  LCOM  expanded  the 
search  if  a  crew  from  the  primary  AFS  was  not  available.  The  expanded  search  was  for  a  crew 
with  at  least  one  member  of  the  primary  AFS  with  other  members  coming  from  the  secondary 
AFS.  If  a  mixed  crew  from  the  primary  and  secondary  AFSs  were  used,  LCOM  chose  the  task 
performance  time  from  a  distribution  with  a  mean  1.12  times  the  mean  a  crew  wholly  from  the 
primary  AFS  (See  Section  2.2.2).  Thus,  compared  to  runs  of  the  independent  alternative  with  the 
same  shift  manning,  fewer  cross  trained  tasks  should  be  backordered  in  the  composite  unit 
alternative  because  crews  were  more  available,  but  the  average  performance  times  should  be 
somewhat  longer  depending  on  the  relative  use  of  primary  and  mixed  crews.  The  added  flexibility 
of  the  composite  unit  alternative  could  lead  to  a  savings  in  manpower  spaces  if  the  demand  for 
cross  trained  tasks  were  great  enough  and  airmen  were  available  from  the  secondary  AFS. 


3.1.2.  Results  of  LCOM  Runs 

The  first  LCOM  runs  were  made  with  the  independent  alternative  (no  resource 
substitution)  with  unconstrained  manpower  resources.  Each  squadron  of  24  aircraft  had  200 
airmen  for  each  of  two  12  hour  shifts  per  day.  The  resulting  number  of  sorties  flown  set  an  upper 
bound  on  sorties  achievable  when  the  model  was  run  with  more  realistic  manpower  resources. 
The  results  of  one  unconstrained  run  for  30  days  are  shown  in  Table  1.  In  the  unconstrained  run, 
the  ATF  achieved  the  desired  sortie  rate  of  just  over  three  sorties  per  aircraft  per  day,  but  the  F- 
15  achieved  only  about  2.5  sorties  per  aircraft  per  day  because  of  constraints  that  were  built  into 
the  LCOM  model.  These  constraints  could  be  a  function  of  availability  of  spares,  failure  rates,  or 
task  time  distributions.  The  study  chose  to  keep  the  LCOM  models  as  they  were  and  to  accept 
the  upper  bounds  of  3  sorties  per  day  for  the  ATF  and  2.5  for  the  F-15.  The  utilization  rate  for 
maintainers  was  computed  as  the  maintenance  manhours  divided  by  the  available  manhours. 
Because  maintainers  must  perform  other  duties  as  well  as  maintenance,  the  Air  Force  does  not 
plan  for  100%  utilization.  For  example.  Tactical  Air  Command  (TAC)  used  83%  as  the  ceiling 
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for  planned  utilization  rates.  Available  hours  were  12  hours  per  day  for  30  days  for  each  member 
of  a  shift.  The  utilization  rates  were  predictably  small  for  the  unconstrained  run. 

Due  to  resource  limitations,  multiple  LCOM  runs  were  made  for  only  a  few  options. 
Since  Monte  Carlo  simulations  such  as  LCOM  rely  on  random  draws  to  determine  outcomes,  the 
results  of  a  number  of  runs  should  be  used  to  estimate  expected  outcomes.  The  results  in  Table  1 
are  from  single  runs,  although  results  for  the  options  that  had  multiple  runs  were  very  consistent. 

Several  different  manning  levels  for  the  two  shifts  were  run  for  both  the  independent  and 
the  composite  alternatives.  For  the  first  independent  run  listed  in  Table  1,  each  shift  for  each 
aircraft  was  manned  at  the  minimum  manning  level,  that  is,  the  number  of  airmen  in  the  largest 
crew  size  for  any  task,  i.e.,  two  for  each  aircraft.  With  this  manning,  the  F-15  sorties  fell  to  1310 
or  1.82  sorties  per  aircraft  per  day.  Also  the  utilization  rate  for  the  F-15  maintainers  climbed  to 
85.3%  which  exceeds  the  ceiling  used  by  TAC.  Reducing  the  manning  for  ATF  maintainers  did 
not  affect  the  number  of  ATF  sorties  flown.  The  R&M  improvements  for  the  ICNIA  systems  of 
the  ATF  reduced  the  workload  to  the  point  where  minimum  manning  was  sufficient  to  handle 
maintenance  without  undue  delays. 

Increasing  the  F-15  manning  to  three  on  the  first  shift  for  the  second  independent  run 
yielded  1400  F-15  sorties  for  a  rate  of  1.94  per  aircraft  per  day.  The  utilization  rate  declined  to 
73.8%,  well  under  the  ceiling.  Note  that  the  difference  in  the  manhours  and  utilization  rates  for 
the  ATF  maintainers  between  independent  runs  one  and  two  should  disappear  if  the  results  of 
several  runs  were  averaged  for  each  column. 

Other  small  increases  in  the  F-15  manpower  did  not  increase  the  number  of  sorties  flown. 
Of  course,  the  increases  reduced  the  utilization  rate  because  more  manhours  were  available. 
Thus,  the  best  manning  for  the  independent  alternative  was  the  manning  in  the  second  independent 
run  and  F-15  performance  was  capped  at  just  under  two  sorties  per  day  per  aircraft,  which  is  less 
than  the  performance  goal. 
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Table  1.  LCOM  Results  for  30  Days  and  24  F-15  and  24  ATF  Aircraft 


Unconstrained 

Independent 

1 

Independent 

2 

Composite  1 

Composite  2 

FI  5  Sorties 

1784 

1310 

1400 

1400 

1388 

F15  Sorties/day 

2.48 

1.82 

1.94 

1.94 

1.93 

ATF  Sorties 

2209 

2209 

2209 

2206 

2209 

ATF  Sorties/day 

3.07 

3.07 

3.07 

3.06 

3.07 

F15  Manhours 

1603 

1229 

1329 

1208 

1184 

ATF  Manhours 

823 

984 

894 

1027 

883 

FlStoATF 

Manhours 

- 

- 

56 

76 

ATFtoFlS 

Manhours 

- 

- 

.  . 

205 

165 

F15  Shift  1 

Manpower 

200 

2 

3 

2 

3 

F15  Shift  2  - 

Manpower 

200 

2 

2 

2 

2 

ATF  Shift  1 

Manpower 

200 

2 

2 

2 

2 

ATF  Shift  2 

Manpower 

200 

2 

2 

2 

2 

F15  Utilization 

Rate% 

1.1 

85.3 

73.8 

83.9 

65.8 

ATF  Utihzation 

Rate% 

.6 

68.3 

62.1 

71.3 

61.3 

The  first  composite  run  used  two  ainnen  for  each  shift  for  each  aircraft.  That  manning 
allowed  the  F-15  to  achieve  1400  sorties,  the  same  as  the  second  independent  run.  F-15 
maintainers  contributed  56  manhours  to  ATF  maintenance  and  ATF  maintainers  spent  205  hours 
on  the  F-15.  There  was  a  net  transfer  of  149  hours  from  the  ATF  to  the  F-15  through  cross 
utilization.  The  utilization  rate  for  F-15  maintainers  was  83.9%,  right  at  the  TAC  ceiling.  Cross 
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utilization  of  the  ATF  maintainers  increased  their  utilization  rate  over  that  for  the  first  independent 
ruii. 


The  second  composite  run  used  three  airmen  for  the  first  F-15  shift,  two  for  the  second, 
and  two  for  each  shift  for  the  ATF.  The  slight  dip  in  F-15  sorties  over  the  first  composite  run  is 
due  to  the  nuances  of  the  random  draws.  If  the  study  had  used  a  number  of  runs  and  averaged  the 
results,  the  achieved  sorties  for  the  ATF  should  be  virtually  the  same  in  all  columns  of  Table  1. 
The  added  manpower  for  the  second  run  did  not  affect  the  number  of  sorties  flown;  it  just  reduced 
the  utilization  rates  for  both  F-15  and  ATF  maintainers.  The  best  manning  for  the  composite 
alternative  was  two  airmen  per  shift  for  each  aircraft  (the  first  composite  run),  although  the  F-15 
utilization  rate  was  at  its  ceding. 

A  number  of  interesting  ideas  could  be  explored  in  the  future.  For  instance,  runs  could  be 
made  with  less  than  minimum  manning  for  each  aircraft  on  at  least  one  of  the  shifts  to  determine 
whether  cross  utilization  could  compensate.  In  this  case,  cross  utilization  would  not  just  be 
optional,  but  would  be  required  for  tasks  with  maximum  crew  size.  Another  idea  is  to  have  one 
way  cross  utilization  with  airmen  from  the  newer,  more  reliable  aircraft  providing  maintenance  on 
the  older  aircraft.  In  this  study,  this  approach  would  allow  the  ATF  maintainers  to  work  on  the  F- 
15,  but  would  restrict  the  F-15  maintainers  to  just  the  F-15.  Since  many  of  the  first  group  of  ATF 
maintainers  would  likely  come  from  the  F-15,  the  cross  training  might  cost  even  less  than 
estimated  for  this  study.  A  third  idea  is  to  study  the  effect  of  the  number  of  cross  trained  tasks  on 
the  manpower  savings.  Cross  utilization  provides  opportunities  to  apply  unused  capacity  for  one 
aircraft  to  the  other,  and  the  more  tasks  that  are  cross  trained,  the  more  opportunities  there  should 
be  to  cross  utilize  the  airmen.  Ultimately,  as  the  number  of  cross  trained  tasks  increases,  the  two 
AFSs  would  essentially  become  one. 


3.2.  Training  Requirements 

The  study  linked  training  requirements  to  items  of  equipment  specified  by  two  or  three- 
digit  WUC.  The  primary  measure  of  training  requirements  was  hours  of  training  by  setting 
including  technical  training  center  (initial  skills  training),  field  training  detachment  (FTD),  and  on- 
the-job  training  (OJT). 
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3.2.1.  Structure  of  Cross  Training 


The  study  concentrated  on  estimating  cross  training  costs  for  the  F-15  because  its  training 
data  were  readily  available.  Training  data  were  more  difficult  to  obtain  and  to  analyze  for  the 
ATF  which  had  a  Navy  aircraft,  F-18,  as  its  BCS.  The  study  assumed  cross  training  for  the  ATF 
was  similar  in  structure,  length,  and  cost  to  that  for  the  F-15. 

During  the  time  period  of  the  study,  training  for  the  primary  AFS  for  F-15  Comm/Nav/Pen 
Aids  flightline  maintenance,  452X1 C,  consisted  of  an  initial  skills  course  at  Lowry  AFB,  an  FTD 
course  at  Eglin  AFB,  and  OJT.  According  to  the  Occupational  Survey  Report  for  AFSC  452X1 C 
(USAFOMC,  1990b)  the  initial  skills  course  lasted  110  days  including  a  42-day  phase  covering 
electronic  principles  and  a  68-day  phase  on  F-15  avionic  communications,  and  the  FTD  course 
lasted  19  days.*^  Airmen  were  granted  skill  level  three  when  they  completed  the  FTD  course. 
OJT  requirements  for  the  452X1 C  were  based  on  the  Specialty  Training  Standard  and  any 
additional  requirements  levied  by  the  unit.  OJT  requirements  were  specified  by  skill  level  starting 
with  the  requirements  for  upgrade  to  5-level.  The  study  assumed  that  training  for  the  primary 
AFS  for  the  ATF,  452XXC,  was  similar. 

The  Air  Force  conducts  cross  training  in  several  settings  including  technical  training 
center,  FTD,  and  OJT.  However,  based  on  comments  from  a  number  of  Air  Force  maintenance 
training  subject  matter  experts  (SMEs)  interviewed  during  the  study,  the  options  for  cross  training 
only  a  portion  of  the  tasks  for  a  secondary  aircraft  were  either  a  combination  of  FTD  and  OJT  or 
just  OJT.^^  They  thought  teaching  a  small  course  like  the  one  proposed  in  the  study  at  a  technical 
training  center  was  too  expensive. 

If  the  study  only  used  OJT  for  cross  training,  all  of  the  training  identified  had  to  be 
converted  to  OJT,  thus  raising  the  issue  of  whether  the  length  of  the  training  changed  as  a  result. 
Some  SMEs  suggested  that  if  initial  skills  or  FTD  training  were  moved  to  OJT,  the  length  of  the 
training  should  be  increased  because  they  thought  OJT  instructors  are  not  as  efficient  as  the 


As  of  15  April  1992,  the  FTD  course  was  eliminated  and  all  material  was  taught  at  the  technical  training 
center  at  Lowry  in  a  94-<iay  course  consisting  of  a  42-day  electronic  principles  phase  and  a  52-day  avionics 
communications  phase.  Simulators  and  aircraft  were  moved  to  Lowry  to  support  the  new  course. 

One  training  planner  from  HQ  TAG  suggested  that  the  material  from  the  initial  skills  and  FTD  courses 
should  be  taught  in  OJT  using  integrated  courseware  (ICW)  that  includes  video  presentations  supporting  an 
interactive  environment  for  practicing  maintenance  skills. 
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instructors  at  a  technical  training  center  or  an  FTD.  They  recommended  inflating  training  hours 
that  were  converted  to  OJT  from  courses  taught  at  a  technical  training  center  or  FTD. 

Another  approach  to  converting  training  between  settings  can  be  based  on  the 
methodology  developed  in  AL/HRT's  Training  Decisions  System  (TDS)  research  project.  TDS 
defines  four  training  settings;  technical  training  center,  FTD,  OJT,  and  Career  Development 
Course;  and  four  types  of  training:  classroom,  small  group  hands-on  training,  hands-on  experience 
on  the  job,  and  self  study.  TDS  used  SME  input  to  develop  training  allocation  curves  to  convert 
training  hours  from  one  type  of  training  to  another.  By  examining  the  proportion  of  each  type  of 
training  for  an  AFS  at  each  setting,  one  can  use  the  allocation  curves  to  convert  training  from  one 
setting  to  another.  However,  TDS  allocation  curves  differ  by  AFS  and  have  been  developed  for 
only  a  few  AFSs.  To  use  them,  the  study  would  have  had  to  assume  curves  from  another  AFS 
applied  to  AFSC  452X1 C  and  to  gather  data  on  the  proportion  of  each  type  of  training  that  was 
in  each  452X1 C  course. 

The  study  chose  to  use  a  combination  of  FTD  and  OJT  for  cross  training  in  order  to 
simplify  the  process  of  converting  training  from  one  setting  to  another.  The  study  assigned  the 
necessary  portions  of  initial  skills  and  FTD  training  to  a  FTD  cross  training  course  and  the  OJT 
portions  to  OJT  cross  training  requirements.  Rather  than  using  allocation  curves  to  estimate  the 
length  of  FTD  training  that  was  converted  from  the  initial  skills  course,  the  study  assumed  that 
the  t3q)e  of  training  would  remain  the  same  at  the  new  setting  (e.g.  topics  that  were  taught  in  the 
classroom  at  the  center  would  also  be  taught  in  the  classroom  at  FTD);  therefore,  initial  skill 
course  hours  were  converted  one-for-one  to  hours  in  FTD.  FTD  and  OJT  hours  did  not  need  to 
be  converted  because  the  cross  training  course  used  the  same  setting  and  type  of  training  that  was 
used  for  the  primary  AFS. 


3.2.2.  Identifying  Cross  Training  Requirements 

Air  Education  &  Training  Command  (AETC)  produces  a  syllabus  or  Plan  of  Instruction 
(POI)  for  the  courses  it  teaches.  The  POI  lists  the  topics  taught  in  the  course  and  the  course 
hours  and  supervised  self-study  hours  allocated  to  each  topic.  The  POIs  for  the  initial  skills  and 
FTD  courses  for  452X1 C  airmen  were  the  starting  points  for  identifying  the  cross  training 
requirements  for  skills  and  knowledge  taught  in  those  courses. 
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The  study  explored  the  possibility  of  using  information  from  the  Training  Extract 
(USAFOMC,  1990a)  produced  by  the  USAF  Occupational  Measurement  Center  (USAFOMC)  as 
a  part  of  the  OS  process.  The  452X1 C  Training  Extract  contained  mappings  of  OS  tasks  to 
topics  on  the  syllabus  for  each  course.  These  mappings,  which  were  done  by  instructors  for  each 
course,  exhibited  problems  similar  to  other  mappings  examined  by  the  study.  Many  of  the  topics 
in  each  POI  had  no  OS  tasks  mapped  to  them,  and  some  of  the  topics  had  multiple  OS  tasks 
mapped  to  them.  The  study  considered  using  the  mapping  of  OS  tasks  to  LCOM  tasks  in 
conjunction  with  the  mapping  of  OS  tasks  to  the  POI  to  relate  training  hours  to  LCOM  tasks. 
However,  the  lack  of  one-to-one  mappings  between  OS  tasks  and  the  POI  and  between  OS  and 
LCOM  tasks  caused  the  study  to  look  for  other  options. 

In  trying  to  resolve  the  many-to-one  and  one-to-many  relationships  identified  in  the  OS  to 
LCOM  mappings,  the  study  looked  at  aggregating  OS  tasks  and  LCOM  tasks  to  try  to  reduce  the 
mappings  to  one-to-one.  The  results  of  the  aggregation  often  led  to  combinations  of  all  tasks 
associated  with  a  three-digit  or  two-digit  WUC  set  or  system.  Rather  than  attempting  to  redo  the 
OS  to  POI  and  OS  to  LCOM  mappings  based  on  the  aggregations,  the  study  investigated 
mapping  topics  from  the  POI  directly  to  three-digit  and  two-digit  WUC  sets  or  systems. 

The  study  presented  the  POI  for  the  initial  skills  course  to  SMEs  and  asked  them  to 
identify  topics  that  were  needed  to  perform  maintenance  on  the  UHF  and  IFF  systems  and  that 
would  not  be  taught  in  the  initial  skills  course  for  the  primary  aircraft,  the  ATF.  General 
knowledge  and  skills  related  to  the  two  systems  would  not  need  to  be  covered  again  in  cross 
training.  Only  those  aspects  of  the  F-15  UHF  and  IFF  systems  that  were  different  from  similar 
systems  on  the  ATF  would  be  included  in  the  cross  training  course.  The  SMEs  used  percentages 
to  indicate  the  proportion  of  the  listed  training  hours  they  thought  were  required  for  the  cross 
training  course.  For  instance,  they  might  have  indicated  that  25%  of  the  hours  needed  to  be  cross 
trained  when  a  topic  was  mostly  general  knowledge  with  some  equipment  specific  aspects. 

The  study  presented  the  POI  for  the  initial  skills  course  to  four  SMEs  and  asked  them  to 
estimate  the  number  of  hours  required  to  teach  UHF  and  IFF  related  topics  from  the  F-15  initial 
skills  course  to  AFSC  452XXC  airmen  who  were  already  trained  to  the  five  level  on  the  ATF. 
The  POI  for  the  initial  skills  course  listed  both  classroom  and  supervised  self-study  hours  for  each 
topic.  The  average  of  the  four  SME  estimates  indicated  that  49.6  classroom  hours  and  18.2  hours 
of  supervised  self-study  should  be  included  in  the  cross  training  course.  These  estimates  were 
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about  25%  of  the  200.5  hours  of  classroom  time  and  68  hours  of  supervised  self-study  time  listed 
in  the  POL  The  survey  results  are  presented  in  Appendix  F. 

The  study  also  presented  the  POI  for  the  FTD  course  to  the  four  SMEs  and  asked  them  to 
identify  topics  required  for  cross  training  ATF  ICNIA  maintainers  to  maintain  the  F-15  UHF  and 
IFF  systems.  The  average  of  the  four  estimates  was  28.2  hours,  again  about  25%  of  the  112 
hours  listed  in  the  POI.  See  Appendix  F  for  more  detail  on  the  survey  results. 

It  seems  reasonable  to  assume  that  the  percentage  of  a  course  identified  for  cross  training 
is  a  function  of  a  number  of  factors  including  the  percentage  of  the  systems  and  sets  maintained  by 
the  AFS  that  are  identified  for  cross  training  (which  is  assumed  to  be  proportional  to  the 
percentage  of  the  KSAs  that  must  be  trained  for  the  AFS)  and  the  relative  emphasis  on  general 
knowledge  and  skills  versus  equipment  specific  knowledge  and  skills.  The  second  factor  is 
important  because  the  cross  training  students  would  already  have  received  training  on  general 
knowledge  and  skills  while  being  trained  for  their  primary  AFS.  Given  the  relation  assumed 
above,  the  consistency  of  the  percentages  across  the  initial  skills  and  FTD  courses  was  a  little 
surprising.  Several  people  interviewed  during  the  course  of  the  study  indicated  that  initial  skills 
courses  for  avionics  were  weighted  toward  general  knowledge  and  skills,  while  the  FTD  courses 
were  weighted  toward  equipment  specific  training  requiring  simulators,  mock-ups  and  equipment. 

This  difference  between  the  two  courses  should  have  caused  a  higher  percentage  of  the  FTD 
course  to  be  identified  for  cross  training,  but  the  study  did  not  find  a  difference. 

Identifying  cross  training  requirements  for  F-15  OJT  was  more  difficult  because  OJT  is 
less  standardized  than  initial  skills  and  FTD  training.  Some  AFSs  have  an  Air  Force  Job 
Qualification  Standard  (JQS)  which  lists  tasks  that  must  be  trained  for  an  airman  to  be  upgraded 
to  the  5,  7,  or  9-level,  but  AFS  452X1C  did  not  have  a  JQS.  When  there  is  no  JQS,  tasks  that 
should  be  trained  in  OJT  are  identified  in  the  Specialty  Training  Standard  (STS)  for  the  AFS. 
These  tasks  are  listed  on  an  airman's  OJT  record.  Air  Force  Form  623.  The  unit  decides  what 
level  of  mastery  of  each  task  is  required  for  upgrade  to  the  5,  7,  or  9-level  and  may  also  add  tasks 
to  the  OJT  record  that  are  not  on  the  STS.  Thus  OJT  can  assume  a  local  flavor,  differing  fi-om 
one  unit  to  another.  Also,  the  OJT  record  does  not  assign  a  training  time  to  each  task.  Instead, 
airmen  are  trained  to  the  level  of  proficiency  or  mastery  specified  in  the  OJT  record  for  upgrade 
to  the  next  skill  level.  The  OJT  instructor  is  responsible  for  determining  when  the  airman  has 
achieved  the  required  proficiency. 
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The  study  used  two  approaches  for  identifying  OJT  requirements  for  cross  training,  a  top- 
down  approach  and  a  bottom-up  approach.  The  top-down  approach  first  asked  for  the  elapsed 
time  spent  in  OJT  upgrade  training  to  the  5-level.  It  then  asked  for  the  percentage  of  each  day  in 
the  period  that  time  that  was  actually  spent  in  OJT.  It  also  asked  for  the  percentage  of  the  OJT 
time  that  was  spent  on  UHF  and  on  IFF  systems,  and  finally  for  the  percentage  of  the  time  on  the 
UHF  and  IFF  systems  that  was  spent  on  each  three-digit  WUC  set  in  the  respective  system.  The 
second,  bottom-up  method  asked  each  SME  to  identify  tasks  on  the  OJT  record  that  were 
required  for  performing  maintenance  on  the  UHF  and  IFF  systems.  It  then  asked  the  SMEs  to 
estimate  the  training  time  spent  on  each  of  the  tasks.  To  help  estimate  the  costs  of  OJT,  the  study 
also  asked  the  one  OJT  instructor  surveyed  to  estimate  the  student  to  instructor  ratio  for  OJT. 
The  study  obtained  estimates  fi’om  two  SMEs  for  both  the  top-down  and  the  bottom-up 
approaches. 

The  results  of  the  top  down  survey  indicated  that  a  typical  upgrade  period  is  12  months 
with  about  30%  of  the  airman's  time  being  spent  in  OJT.  The  percentage  of  OJT  devoted  to  the 
UHF  system  was  10%;  the  percentage  devoted  to  the  IFF  system  was  25%.  See  Appendix  F  for 
detailed  survey  results  including  a  breakout  by  three-digit  WUC.  When  the  SME  percentages 
were  applied  to  the  upgrade  period,  the  average  estimate  of  OJT  time  was  57  hours  for  the  UHF 
system  and  146  hours  for  the  IFF  system. 

The  results  of  the  bottom-up  approach  indicated  that  about  35  tasks  on  the  OJT  record 
should  be  included  in  cross  training.  The  SMEs  estimated  that  38.75  hours  of  UHF  OJT  and  68.5 
hours  of  IFF  OJT  were  needed  for  cross  training.  The  study  chose  to  use  the  top-down  results 
over  those  from  the  bottom-up  approach  for  several  reasons.  The  study  gathered  data  from  more 
SMEs  for  the  top-down  approach,  providing  more  confidence  in  the  results.  Also,  the  lack  of 
benchmark  times  for  individual  tasks  or  topics,  such  as  the  POI  provided  for  the  other  estimates, 
contributed  to  a  lack  of  consistency  in  the  estimates  for  the  two  SMEs  polled.  See  Appendix  F 
for  more  details. 

The  study's  limited  survey  of  SMEs  yielded  training  requirements  data  with,  in  some  cases, 
wide  variability.  However,  the  primary,  issue  here  was  not  developing  precise  training 
requirements  based  on  SME  surveys.  Rather,  the  study  concentrated  on  the  SME  estimation 
process  to  see  if  the  approach  was  a  viable  one.  In  all  cases,  the  SMEs  were  comfortable  and 
confident  estimating  training  requirements.  However,  the  validity  of  their  estimates  should  be 
evaluated  before  a  large  data  collection  effort  is  begun.  The  study  concluded  that  SME  surveys 
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are  an  acceptable  means  for  collecting  training  requirements  data  if  the  surveys  are  precisely 
worded  and  if  a  sufficiently  large  sample  of  SMEs  is  polled.  Section  4.2,  Training  Costs, 
discusses  the  process  used  to  convert  these  estimates  of  initial  skills  course  hours,  FTD  hours,  and 
OJT  hours  to  training  costs  for  a  cross  training  course. 
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4.  LIFE-CYCLE  COSTS 


4.1.  Manpower  Costs 

The  study  estimated  the  costs  of  the  differences  in  manpower  between  the  two 
alternatives.  To  make  the  estimates,  the  study  modified  the  Army  Manpower  Cost  System 
(AMCOS)  model,  and  then  used  the  modified  model,  the  Air  Force  Manpower  Cost  System 
(AFMCOS),  to  estimate  manpower  costs.  The  primary  modifications  were  to  replace  Army  data 
with  Air  Force  data  and  to  eliminate  options  that  only  applied  to  the  Army.  The  next  few  sections 
briefly  describe  the  LCC  methodology  and  data  used  in  AFMCOS  and  highlight  the  revisions 
made  to  AMCOS  to  produce  this  first  version  of  AFMCOS. 


4.1.1.  Costs  Included  in  AFMCOS 

AFMCOS  had  several  cost  policy  modules,  each  of  which  estimates  a  certain  category  of 
costs.  These  cost  policy  modules  included  the  following: 

•  military  compensation 

•  retired  pay  accrual 

•  selective  reenlistment  bonus 

•  special  pays 

•  training 

•  recruiting 

•  medical  support 

•  other  benefits 

•  permanent  change  of  station  (PCS) 

•  officer  acquisition 

•  GIBill 

In  general,  each  module  computed  an  average  cost  for  each  possible  grade  of  airmen  and  officers. 
In  some  cases  the  averages  were  further  broken  out  by  AFS.  Average  costs  were  computed  for 
two  budget  categories;  military  personnel  account  (MPA)  and  operations  and  maintenance 
(O&M).  Each  of  the  modules  is  discussed  in  turn. 
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Military  compensation  consisted  of  all  variable  costs  that  provide  basic  pay,  basic 
allowance  for  quarters  (BAQ),  basic  allowance  for  subsistence  (BAS),  and  variable  housing 
allowance  (VHA).  Average  basic  pay  was  determined  for  each  grade  as  a  weighted  average  of 
the  basic  pay  for  each  year  of  service  (YOS)  for  the  grade  using  the  fraction  of  the  inventory  in 
the  YOS  for  the  grade  as  the  weight.  Average  BAQ  was  determined  for  each  grade  as  a  weighted 
average  of  the  with  and  without  dependents  rates,  where  the  weights  are  the  fraction  receiving 
each  rate.  Average  BAQ  is  further  weighted  by  multiplying  by  the  fraction  receiving  BAQ  in-cash 
(as  opposed  to  BAQ  in-kind  for  those  who  live  in  quarters  provided  by  the  government). 
Average  BAS  for  each  grade  was  assumed  to  be  the  in-cash  rate  for  the  grade  even  though  some 
members  receive  BAS  in-kind.  Average  VHA  for  each  grade  was  computed  as  the  weighted 
average  of  the  VHA  for  each  location  where  the  weights  were  the  fraction  of  the  grade  inventory 
at  the  location.  Average  VHA  was  weighted  further  by  the  fraction  of  the  grade  receiving  BAQ 
in-cash.  Weights  used  in  AFMCOS  were  taken  from  AMCOS  and  thus  were  based  on  weights 
for  the  Army. 

Retired  pay  accrual  cost  was  determined  for  each  grade  as  the  average  basic  pay  per 
grade  multiplied  by  a  cost  percentage  rate  obtained  from  the  DoD  actuary.  An  alternate  estimate 
available  in  AMCOS  for  use  by  the  Army  did  not  apply  to  the  Air  Force  and  thus  was  not 
included  in  AFMCOS. 

Selective  reenlistment  bonuses  were  a  function  of  the  award  level  or  multiplier,  years  of 
reenlistment,  and  basic  pay.  Current  multipliers  were  included  in  AFMCOS  for  the  AFSC  of 
interest  in  the  study,  452X1C,  and  were  assumed  to  apply  to  the  new  AFSC,  452XXC,  as  well. 
The  average  basic  pay  for  each  grade  and  average  length  of  reenlistment  were  used  in  the 
computation  of  the  average  reenlistment  bonus  for  each  grade.  Average  basic  pay  and  average 
length  of  reenlistment  were  taken  from  AMCOS  (Army)  data.  The  actual  costs  of  selective 
reenlistment  bonuses  were  weighted  by  the  probability  that  members  receive  the  bonuses. 
Bonuses  did  not  affect  the  proof  of  concept  study  because  the  multipliers  for  AFSC  452X1 C  were 
zero. 

Special  pays  included  a  number  of  incentive  pays  such  as  hazardous  duty,  overseas 
allowance,  and  special  duty  assignment  pay.  AFMCOS  used  a  separate  computation  to  determine 
the  average  of  each  special  pay  by  grade  and  AFS.  These  computations  determined  an  average 
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pay  assuming  the  pay  was  received  and  then  weighted  it  by  the  probability  of  receiving  the  pay. 
Overseas  allowance  was  the  only  special  pay  used  in  the  proof  of  concept  study. 

Training  costs  included  the  costs  of  Basic  Military  Training  (BMT),  initial  skills  training, 
FTD  training  to  award  skill  level  three,  skill  progression  training,  undergraduate  flight  training, 
cost  of  training  officer  accessions,  and  the  cost  of  professional  military  education  (PME).  The 
costs  of  BMT  and  the  initial  skills  and  FTD  courses  for  AFSC  452X1C  were  taken  from  the  ATC 
Cost  Factors  report  (ATC,  1990).  All  other  training  costs  were  based  on  Army  data  from 
AMCOS.  Because  the  proof  of  concept  study  did  not  consider  officer  positions,  the  only  training 
costs  based  on  Army  data  used  in  the  study  were  the  costs  of  skill  progression  training  and  PME. 

Recruiting  costs  were  computed  per  recruit  using  data  provided  by  Air  Force  Recruiting 
Service  (RS)  to  determine  the  costs  of  recruiters,  recruit  processing  costs,  and  RS  operations 
costs.  AFMCOS  used  DoD  data  to  determine  the  Air  Force  share  of  "joint  costs"  including 
advertising,  market  research,  and  recruiting  facilities.  AFMCOS  did  not  compute  separate 
recruiting  costs  for  high  and  low  quality  recruits  as  was  done  in  AMCOS,  as  all  recruits  were 
considered  to  be  high  quality. 

Medical  support  was  computed  as  the  sum  of  the  costs  of  CHAMPUS  and  the  costs  of 
operating  Air  Force  health  care  facilities.  Average  CHAMPUS  costs  per  grade  were  computed 
from  the  average  costs  of  CHAMPUS  per  active  duty  dependent  times  the  average  number  of 
dependents  for  the  grade.  The  costs  of  the  Air  Force  medical  force  and  facilities  were  attributed 
to  their  wartime  support  mission  so  only  the  costs  of  operating  the  health  care  facilities  were 
accounted  for  in  AFMCOS.  These  costs  were  prorated  to  each  eligible  person,  members  and 
dependents,  and  then  were  allocated  by  grade  as  a  function  of  the  average  number  of  dependents 
for  members  in  the  grade.  Army  data  from  AMCOS  were  used  in  AFMCOS. 

Other  benefits  included  separation  pay,  clothing  allowance,  the  government  contribution 
to  social  security  tax,  survivor's  benefit,  MWR  (morale,  welfare,  and  recreation)  benefits,  and 
miscellaneous  costs.  The  average  costs  of  each  of  these  benefits  were  computed  for  each  grade 
based  on  the  weighted  average  of  their  use  and  cost  for  the  grade.  Army  data  from  AMCOS  were 
used  in  AFMCOS. 

Permanent  change  of  station  (PCS)  costs  were  estimated  for  five  categories  of  moves; 
rotational,  operational,  accession,  training,  and  separation  moves.  The  average  cost  of  each 
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category  of  moves  was  taken  from  the  USAF  Cost  and  Planning  Factors  report  (AFCSTC/OS, 
1989).  Air  Force  data  on  tour  lengths  and  the  distribution  of  452X1C  positions  overseas  and  in 
CONUS  were  used  to  determine  the  probability  of  a  move.  Army  data  from  AMCOS  on  the 
fraction  of  each  grade  which  has  dependents  were  used  to  estimate  the  effect  of  weight 
allowances  on  PCS  costs. 

Officer  acquisition  costs  included  costs  such  as  advertising,  scholarships,  initial  training, 
military  pay  and  allowances  for  cadets  and  officer  trainees,  and  operations  and  support  costs  for 
the  Air  Force  Academy,  Officer  Training  School,  and  Reserve  Officer  Training  Corps.  Average 
costs  were  computed  per  new  officer  using  Army  data  from  AMCOS.  However,  no  officer 
positions  were  considered  in  the  proof  of  concept  study. 

GI  Bill  costs  are  funded  by  the  Veteran's  Administration  and  thus  may  be  suppressed  for 
some  applications  of  the  model.  The  basic  benefit  is  $300  per  month  for  up  to  36  months. 
Average  costs  were  computed  from  expected  usage  rates  and  expected  costs  for  each  participant. 

Army  data  from  AMCOS  were  used  in  AFMCOS. 


4.1.2.  Cost  Computation 

As  airmen  and  officers  flow  through  the  personnel  inventory,  certain  "investment"  or  "one¬ 
time"  costs  are  incurred.  These  costs,  such  as  the  initial  costs  of  recruiting  and  training  airmen, 
benefit  the  Air  Force  throughout  the  member's  career.  AFMCOS  amortizes  these  costs  over  the 
expected  years  of  service  for  the  member.  The  proof  of  concept  study  used  data  for  AFSC 
452X1 C  to  compute  expected  years  of  service.  With  this  treatment  of  "investment"  costs,  all 
costs  become  an  average  cost  per  position  or  person  per  year. 

AFMCOS  computed  the  costs  for  each  year  of  the  life  of  the  weapon  system  using  the 
manning  structure  for  each  unit  and  the  fielding  plan  for  the  number  of  units  for  each  year  of  the 
system  life  assumed  for  the  study.  Standard  DoD  inflation  rates  (around  4%)  were  used  to  adjust 
costs  for  future  years.  Standard  DoD  discount  rates  (10%)  were  then  used  to  compute  the 
present  value  of  all  costs  over  the  life  of  the  system.  All  costs  were  converted  to  1992  dollars. 
AFMCOS  includes  an  option  that  allows  the  user  to  directly  compare  the  costs  of  two 
alternatives.  The  study  used  this  option  to  compare  the  independent  and  composite  maintenance 
alternatives  studied. 
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4.1.3.  Results 


The  LCOM  results  presented  in  Table  1  were  for  units  of  24  F-15  and  24  ATF  aircraft. 
The  study  assumed  units  were  fielded  according  to  the  plan  presented  in  Table  2.  All  units  were 
retired  by  year  28. 


Table  2.  Fielding  Plan  for  ATF  and  F-15  Squadrons 


Year 

Units 

Year 

Units 

Year 

Units 

1 

1 

10 

9 

19 

9 

2 

2 

11 

9 

20 

9 

3 

3 

12 

9 

21 

8 

4 

4 

13 

9 

22 

7 

5 

5 

14 

9 

23 

6 

6 

6 

15 

9 

24 

5 

7 

7 

16 

9 

25 

4 

8 

8 

17 

9 

26 

3 

9 

9 

18 

9 

27 

2 

As  noted  in  Section  3.1.2,  the  best  manning  for  the  independent  alternative  was  to  assign 
two  airmen  to  each  shift  for  each  aircraft  except  for  one  of  the  F-15  shifts  which  had  three  airmen 
assigned.  Also,  the  best  option  for  the  composite  unit  was  to  assign  only  two  airmen  for  each 
aircraft  for  each  shift.  Thus  the  composite  unit  needed  only  four  airmen  compared  to  the  five 
needed  for  the  independent  unit.  Manpower  standards  were  used  to  convert  positions  from  the 
model  into  manpower  spaces.  The  manpower  standard  used  permits  244  hours  per  space  per 
month.  For  the  best  independent  option,  the  standard  allowed  eight  spaces  for  the  F-15  and  six 
for  the  ATF.  For  the  best  composite  option,  the  standard  allowed  six  spaces  for  each  type  of 
aircraft.  When  nine  squadrons  of  each  aircraft  were  fielded,  there  was  a  difference  of  18  spaces 
between  the  two  alternatives  with  the  composite  alternative  having  fewer.  Temporarily  ignoring 
the  additional  cost  of  cross  training  airmen  in  the  composite  unit,  AFMCOS  estimated  the 
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difference  in  the  discounted  costs  of  the  manpower  under  the  two  alternatives  at  $8,038  million 

18 

with  the  composite  alternative  being  cheaper. 


4.2.  Training  Costs 

The  various  components  of  training  costs  can  be  designated  as  fixed  or  variable.  The 
study  concentrated  on  variable  costs  which  usually  represent  the  cost  of  delivering  training  and 
are  presented  as  cost  per  student.  The  study  used  published  sources  for  the  cost  of  ATC  courses 
(HQ  ATC,  1990)  and  developed  its  own  estimates  for  OJT  costs. 

As  with  manpower  costs,  the  study  estimated  the  difference  in  the  training  costs  between 
the  two  alternatives.  The  costs  of  initial  skills  and  FTD  training  for  the  primary  aircraft  were 
included  in  AFMCOS  as  a  part  of  the  estimate  of  the  cost  of  any  differences  in  manpower 
between  the  two  alternatives.  In  addition,  the  study  considered  the  cost  of  cross  training  required 
in  the  second  alternative.  F-15  maintainers  from  AFSC  452X1C  received  cross  training  on  their 
secondary  aircraft,  the  ATF,  and  ATF  maintainers  from  AFSC  452XXC  received  cross  training  on 
their  secondary  aircraft,  the  F-15. 

The  study  assumed  that  airmen  would  be  cross  trained  for  their  secondary  aircraft  after 
they  had  been  upgraded  to  the  5-level  for  their  primary  aircraft.  The  study  also  assumed  that 
cross  training  would  cover  all  the  training  requirements  for  the  5-level  for  the  two-digit  WUC 
systems  selected  for  cross  training. 


4.2.1.  Estimating  Costs  of  Initial  Skills  Training 

The  study  estimated  the  cost  of  cross  training  by  first  estimating  the  cost  of  the  cross 
trained  portions  of  each  course  and  OJT.  To  be  consistent  with  the  manpower  costs,  training 
costs  were  converted  to  1992  dollars  using  standard  DoD  inflation  rates. 

ATC  reported  the  cost  of  the  initial  skills  course,  for  both  the  electronic  principles  and  the 
avionics  phases,  as  $12430  per  graduate  (HQ  ATC,  1990).  As  mentioned  in  Section  3.2,  for  the 
time  period  covered  by  the  study,  the  initial  skills  course  consisted  of  a  42-day  electronic 


For  simplicity,  all  positions  were  designated  as  requiring  grade  E5. 
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principles  phase  and  a  68-day  avionics  phase.  Prorating  the  cost  to  the  cross  training  course 
raised  a  problem  with  accounting  for  all  the  hours  in  the  course.  The  June  1989  POI  used  in  the 
study  only  listed  268.5  hours  (200.5  hours  of  classroom  training  and  68.5  hours  of  supervised 
self-study,  for  a  total  of  33.6  days)  in  the  avionics  phase  instead  of  the  544  hours  expected  for  68 
days  of  8  hours  each.  The  452X1  training  manager  at  Lowry  was  not  familiar  with  the  POI  used 
in  the  study  but  said  that  the  new  POI  accounts  for  all  the  hours  in  the  new  94  day  course  (42 
days  for  electronic  principles  and  52  days  for  avionics  which  now  includes  the  material  formerly 
taught  in  the  FTD  course).  Therefore,  the  study  assumed  that  the  old  POI  accounted  for  all  68 
days  of  avionics  training  and  prorated  the  training  costs  as  follows.  Because  68  of  1 10  total  days 
were  spent  on  avionics,  the  cost  of  the  avionics  phase  was  estimated  to  account  for  68/110  or 
about  62%  of  the  total  of  $12430,  i.e.,  $7684.  As  estimated  in  Section  3.2,  the  UHF  and  IFF 
cross  training  portions  of  the  avionics  phase  accounted  for  67.8  of  268.5  hours  in  the  POI.  Thus, 
cross  training  the  UHF  and  IFF  portions  of  the  course  was  estimated  to  cost  about  25%  of  $7684, 
which  is  $1940. 


4.2.2.  Estimating  Costs  of  Field  Training  Detachment  Training 

The  study  was  unable  to  find  costs  for  the  FTD  course;  however,  the  ATC  Comptroller 
maintains  cost  factors  for  each  week  of  FTD  that  can  be  applied  to  the  length  of  a  given  FTD 
course  to  estimate  its  cost.  Using  the  factor  of  $889  per  week,  the  estimated  cost  of  the  FTD 
course  was  $3379.  Similar  to  problems  encountered  with  the  POI  for  the  initial  skills  course,  the 
POI  for  the  FTD  course  listed  a  total  of  1 12  hours  rather  than  the  expected  152  hours  (19  days  x 
8  hours).  Again  the  study  assumed  that  the  POI  accounted  for  the  entire  course.  According  to 
Section  3.2,  the  28.2  hours  identified  for  cross  training  were  about  25%  of  the  total.  Thus  the 
cost  of  cross  training  the  UHF  and  IFF  portions  of  the  FTD  course  was  estimated  as  25%  of 
$3379,  or  $851. 


4.2.3.  Estimating  Costs  of  On-The-Job  Training 

Estimating  the  costs  of  OJT  is  a  controversial  topic.  Some  of  the  controversy  is  due  to 
the  fact  that  required  maintenance  is  often  performed  while  OJT  is  conducted.  Most  cost  analysts 
think  that  time  spent  performing  required  maintenance  should  not  be  charged  to  OJT.  However, 
the  difficulty  then  becomes  determining  the  crew  size  and  time  required  for  the  maintenance  as 
compared  to  that  for  maintenance  done  in  conjunction  with  OJT.  Routinely  collected 
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maintenance  data  do  not  permit  precise  determination  of  which  maintenance  actions  and  time 
should  be  counted  as  OJT,  nor  do  they  permit  a  precise  determination  of  student  to  instructor 
ratio  for  OJT.  Therefore,  the  study  used  the  estimates  of  time  spent  in  OJT  presented  in  Section 
3.2.  The  study  used  an  average  of  1.5  students  per  instructor  as  the  student  to  instructor  ratio, 
based  on  SME  estimates.  The  study  followed  the  lead  of  TDS  by  computing  the  cost  of  OJT  as 
the  sum  of  the  costs  of  student  time,  instructor  time,  and  expendables  (Rueter,  1989).  The  study 
did  not  identify  any  expendables  for  452X1 C  OJT  and  thus  assumed  that  no  expendables  were 
consumed.  Because  the  study  assumed  that  OJT  would  not  occur  until  the  airmen  had  been 
granted  skill  level  five  in  their  primary  AFS,  OJT  students  were  assumed  to  be  E4s  with  four  years 
of  service.  The  study  assumed  OJT  instructors  were  E5s  with  six  years  of  service.  The  hourly 
costs  depend  on  which  cost  elements  are  included.  The  study  used  cost  elements  included  in 
military  compensation,  money  actually  paid  to  the  airmen  monthly,  to  compute  hourly  costs. 
Military  compensation  includes  basic  pay,  BAQ,  BAS,  and  VHA.*^ 

The  components  of  military  compensation  can  vary  due  to  the  number  of  dependents, 
location,  and  other  factors.  Using  average  values  for  the  components,  the  study  estimated  the 
hourly  costs  to  be  $9.87  for  an  E4  with  four  years  of  service  and  $1 1.36  for  an  E5  with  six  years 
of  service.  Based  on  a  student  to  instructor  ratio  of  1.5,  the  cost  per  hour  of  OJT  is  $9.87  for  the 
student  plus  $7.57  for  the  instructor  (  1/1.5  x  $11.36)  for  a  total  $17.44  per  hour.  Based  on  the 
top-down  estimate  of  57  hours  for  the  UHF  system  and  146  hours  for  the  IFF  system,  the  total 
OJT  cross  training  hours  were  203  hours,  giving  a  cost  per  student  of  $3540  (203  hours  x 
$17.44/hour). 


4.2.4.  Estimating  Total  Training  Cost 

The  total  cost  per  student  of  the  cross  training  course  for  the  F-15  UHF  and  IFF  systems 
was  $1940  for  the  initial  skills  portion,  $851  for  the  FTD  portion,  and  $3540  for  the  OJT  portion 
for  a  total  of  $6331.  The  study  assumed  there  were  no  course  development  costs  or  training 
equipment  purchases.  ^ 


According  to  (Rueter,  1989),  TDS  used  only  basic  pay  in  computing  hourly  costs. 
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Training  data  were  not  readily  available  for  the  Navy's  F-18,  the  BCS  for  the  ATF ICNIA 
systems;  therefore,  the  cost  of  ATF  cross  training  was  not  estimated  directly.  Instead,  the  cost 
was  simply  assumed  to  be  the  same  as  the  cost  for  F- 15  cross  training  or  $633 1  per  student. 

Converting  per  student  costs  to  LCCs  requires  consideration  of  the  flow  of  airmen  into  the 
inventory  for  an  AFS.  Since  this  conversion  process  is  already  embedded  in  the  AFMCOS  model, 
the  study  used  AFMCOS  to  estimate  the  LCCs  of  conducting  F-15  and  ATF  cross  training  by 
creating  two  new  AFSs,  one  for  each  aircraft.  All  data  for  the  two  new  AFSs  were  same  as  those 
for  AFSCs  452X1 C  and  452XXC,  respectively,  except  that  the  cost  of  FTD  was  increased  by 
$6331,  the  cost  of  the  cross  training  course  for  the  other  aircraft.  In  its  computations,  AFMCOS 
prorated  these  costs  over  the  expected  length  of  service  for  an  airman  from  the  AFS  which  was 
based  on  data  for  AFS  family,  452X1,  which  includes  AFSC  452X1 C.  The  study  made  AFMCOS 
runs  with  and  without  the  cross  training  costs  and  compared  them  to  estimate  the  LCC  of  the 
cross  training  courses.  For  the  best  composite  option  and  fielding  plan  used  in  the  study, 
AFMCOS  estimated  the  LCC  of  cross  training  to  be  $872,000. 
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5.  SUMMARY  AND  CONCLUSIONS 


This  section  of  the  report  summarizes  the  findings  of  the  proof  of  concept  study  to 
determine  which  of  the  two  alternatives  has  lower  LCCs.  The  section  also  reaches  conclusions  on 
the  larger  goals  of  the  effort;  1)  to  determine  whether  data  exist  to  support  the  system  of  models 
proposed  for  SYSMOD,  and  2)  whether  the  conceptual  framework  for  SYSMOD  provides  a 
sound  basis  for  estimating  the  MPT  resource  requirements  and  LCCs  of  alternatives  proposed 
during  Concept  Exploration.  Finally,  the  section  discusses  the  steps  that  should  be  taken  in  order 
to  develop  SYSMOD  for  Concept  Exploration. 


5.1.  Summary  of  Comparison  of  the  Two  Alternatives 

The  study  found  the  composite  alternative  to  be  less  costly  than  the  independent 
alternative.  The  composite  alternative  had  a  manpower  savings  of  8.038  million  dollars  which 
more  than  offset  the  additional  cost  of  .872  million  dollars  for  cross  training  yielding  a  net  savings 
of  7.166  million  dollars.  In  fact,  the  manpower  savings  were  sufficient  to  offset  training  costs 
about  nine  times  as  large  as  those  estimated  in  the  study. 


5.2.  Conclusions  on  Goals  for  the  Effort 

The  study  found  SYSMOD  to  be  a  viable  approach  for  estimating  the  MPT  LCC  of 
systems  evaluated  during  Concept  Exploration.  It  found  that  data  can  be  obtained  to  support  the 
models  needed  to  make  the  estimates  and  that  the  proposed  conceptual  framework  is  a  sound  one 
for  the  comparison  of  alternatives. 


5.2.1.  Data  Availability 

Three  categories  of  data  are  required  to  support  the  models  envisioned  for  SYSMOD. 
Reliability  and  maintdnability  (R&M)  data  are  required  to  estimate  the  maintenance  workload  and 
manpower  requirements.  Training  data  are  needed  to  estimate  training  requirements,  particularly 
the  lengths  of  training  courses.  Cost  data  and  factors  are  also  needed  to  estimate  the  LCCs  of 
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meeting  the  manning  and  training  requirements  of  the  proposed  aircraft  and  to  estimate  the  cost  of 
spares  and  support  equipment,  if  necessary. 

In  a  manner  typical  of  Concept  Exploration  studies,  this  study  used  data  from  existing 
aircraft  that  served  as  the  parent  systems  for  the  proposed  aircraft.^®  Fairly  well-developed 
procedures  exist  for  extracting  R&M  data  on  existing  systems.  These  procedures  have  evolved  to 
support  LCOM  modeling  for  manpower  studies.  Previous  studies  have  even  developed  LCOM 
data  for  non-Air  Force  aircraft  such  as  the  Navy  F-18  considered  in  this  study.  Because  the 
LCOM  data  structure  is  well  established  in  the  manpower  community,  the  study  chose  to  use 
LCOM  data  as  its  R&M  data  structure.  The  study  did  not  find  a  ready  source  of  training  data; 
instead,  the  study  developed  its  own  training  data  based  on  existing  Air  Force  data.  The  study 
used  cost  data  from  an  existing  Army  manpower  cost  model,  AMCOS,  as  a  starting  point.  The 
study  sought  sources  of  Air  Force  specific  cost  data  to  replace  some  of  the  Army  cost  data.  The 
study  did  not  attempt  to  identify  sources  for  cost  data  for  spares,  support  equipment,  training 
development,  and  training  equipment. 

After  some  false  starts,  the  study  chose  to  aggregate  data  to  the  three-digit  WUC  or  set 
level.  This  was  the  least  aggregated  (most  detailed)  level  that  the  study  found  to  be  feasible.  The 
study  had  problems  developing  training  requirements  data  in  more  detail  (e.g.  by  task  at  the  three- 
digit  WUC).  In  fact,  some  training  data  can  be  broken  out  better  at  only  the  two-digit  WUC 
level;  however,  the  study  developed  reasonable  methods  for  assigning  the  data  to  the  three-digit 
WUC  level.  Because  R&M  data  can  be  extracted  with  greater  detail  than  the  three-digit  WUC 
level,  aggregating  them  to  the  three-digit  level  is  fairly  straightforward.  Manpower  cost  data 
presented  few  problems  because  manpower  costs  were  only  applied  once  manpower  positions 
were  estimated.  Training  costs  presented  problems  similar  to  those  discovered  Avith  training 
requirements  data.  The  study  developed  methods  for  assigning  the  costs  in  proportion  to 
requirements  at  the  three-digit  WUC  level. 

Reliability  and  Maintainability  Data:  The  study  used  existing  LCOM  models  as 
sources  of  R&M  data  on  the  parent  systems  for  the  study,  the  F-18  and  F-15.  These  LCOM 
models  specified  tasks  to  the  four-digit  WUC  level  and  tracked  individual  resources;  thus,  they 
were  more  detailed  than  the  three-digit  WUC  level  required  for  Concept-Exploration. 
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New  aircraft  are  often  described  as  derivatives  of  an  existing  aircraft  or  parent  system  which  serves  as  a  model 
or  source  of  data  for  early  studies. 
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Training  Data:  The  study  found  that  training  requirements  data  for  initial  skills  and  FTD 
courses  could  be  extracted  from  the  course  POIs.  The  POIs  were  readily  available  from  the  ATC 
training  manager  for  the  courses  or  from  the  Training  Extract  developed  by  USAFOMC.  Based 
on  the  limited  data  collection  effort  conducted  as  a  part  of  the  study,  it  appears  that  airmen  from 
an  AFS  can  associate  training  requirements  with  two  and  three-digit  WUC  systems  and  sets. 
Thus  a  comparison  system  data  base  could  also  contain  training  requirements  data  for  ATC 
courses.  A  review  of  the  SME  responses  indicated  that  a  knowledgeable  analyst  could  have  made 
most  of  the  associations  as  well.  Developing  training  requirements  data  for  OJT  was  more 
difficult  because  OJT  training  requirements  are  less  well  defined  and  are  less  detailed  than  those 
for  initial  skills  and  FTD  courses.  The  study  used  the  OJT  record  as  a  starting  point  for 
developing  OJT  training  requirements  data.  However,  tasks  listed  on  an  OJT  record  vary  with  the 
unit  to  which  an  airman  is  assigned  and  do  not  have  a  specified  length  of  training.  The  study 
explored  two  methods,  top-down  and  bottom-up,  that  could  be  used  to  develop  OJT  training 
requirements  data  based  on  SME  judgments.  The  methods  should  be  refined  before  they  are  used 
to  gather  data  to  stock  a  BCS  data  base  with  training  data  on  parent  systems.  Refinements 
examined  should  include  gathering  data  using  group  consensus  rather  than  averaging  individual 
estimates.  As  shown  in  Appendix  F,  SMEs  differed  on  some  of  the  items  selected  for  cross 
training  Forming  a  group  consensus  may  be  a  better  approach  for  resolving  the  differences. 

Cost  Data:  Collecting  the  manpower  cost  factors  included  in  AMCOS  is  straightforward 
but  time  consuming  because  the  factors  are  found  in  a  number  of  different  documents  or  are 
maintained  in  data  bases  belonging  to  a  number  of  different  offices.  A  one-time  effort  is  required 
to  identify  the  sources  of  the  data;  however,  the  data  themselves  must  be  updated  periodically. 
Training  cost  factors  are  readily  available  from  the  ATC  Cost  Factors  Report  (ATC,  1990).  The 
study  explored  a  method  for  prorating  costs  to  training  hours  that  appeared  to  give  reasonable 
results.  Follow  on  efforts  should  work  with  the  ATC  Comptroller  to  refine  the  methods. 


5.2.2.  Soundness  of  Conceptual  Framework 

For  SYSMOD  to  be  viable,  its  conceptual  framework  must  provide  a  sound  basis  for 
comparing  alternatives.  With  R&M  and  training  data  available  at  a  three-digit  WUC  level  on 
parent  systems,  a  BCS  data  base  can  be  built  to  support  SYSMOD  models.  SYSMOD  could 
include  a  data  editor  that  allows  the  user  to  apply  factors  to  the  R&M  data  to  change  time 
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between  failures,  task  times,  or  crew  size  to  represent  R&M  improvements  over  the  parent 
system.  Similarly,  training  data  could  be  edited  by  applying  factors  to  change  the  length  or  cost  of 
training  over  the  parent  system.  The  changes  might  anticipate  increased  efficiency  in  training 
delivery  or  a  change  in  training  hours  due  to  a  change  in  the  KSAs  that  must  be  trained. 

As  mentioned  earlier,  the  LCOM  models  used  in  the  study  were  more  complicated  than 
required  and  so,  more  cumbersome.  A  simpler  simulation  or  an  analytic  approximation  is 
probably  sufficient  for  estimating  differences  between  proposed  alternatives  when  available  data 
are  aggregated  to  the  three-digit  WUC  level.  The  analytic  methodology  used  in  SUMMA  may 
provide  the  foundation  for  estimating  maintenance  manpower  in  SYSMOD.  Using  analytic 
models  with  a  three-digit  WUC  level  of  detail  together  with  fairly  constant  sortie  requirements 
should  provide  the  consistency  between  models  and  data  required  to  make  good  estimates. 

Basing  the  LCC  model  for  MPT  costs  on  AMCOS  or  AFMCOS  will  provide  a  framework 
for  estimating  the  three  types  of  costs.  Personnel  costs  such  as  those  for  recruiting  and  PCS 
moves  and  training  costs  are  prorated  over  the  member's  expected  service  time  to  provide  a  basis 
for  including  them  in  the  same  model  with  the  continuing  costs  of  manpower.  AFMCOS  is 
flexible  and  easy  to  use  and  can  be  extended  to  include  costs  of  spares,  support  equipment, 
training  development,  and  training  equipment. 

Several  tradeoff  models  appear  to  be  feasible  within  the  SYSMOD  framework.  The  study 
of  trades  between  R&M  factors  and  manpower  could  be  facilitated  through  the  application  of 
R&M  factors  to  parent  system  data  together  with  changing  the  manning  levels  for  the  AFSs.  The 
study  found  that  the  manpower  model  should  allow  variable  shift  manning,  including  shifts  with 
less  than  minimum  manning.  Otherwise,  the  impact  of  R&M  improvements  on  manpower  can  be 
limited  if  the  lower  bound  on  shift  manning  is  the  minimum  manning  level. 

AFS  restructuring  trades  are  also  possible  within  the  SYSMOD  framework  by  shifting 
three-digit  WUC  sets  of  tasks  between  AFSs.  More  work  must  be  done  to  investigate  the  effects 
on  task  performance  time  and  training  requirements,  but  until  more  is  known,  the  model  could 
make  simplifying  assumptions,  such  as  the  task  time  does  not  change  and  training  requirements 
are  independent  and  are  therefore  additive. 

Other  trades  are  readily  included  in  the  SYSMOD  framework,  but  they  depend  on 
extending  model  capabilities  beyond  the  first  version  of  SYSMOD  or  on  additional  data 
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collection  and  analysis  efforts.  For  instance,  trades  involving  changes  in  the  levels  of  maintenance 
depend  on  changes  in  R&M  factors  and  in  the  supply  of  spares.  The  cost  of  spares  must  be 
included  in  the  LCC  model  before  this  trade  is  effective.  Another  trade,  one  involving  experience 
mix  and  manpower  costs  is  dependent  on  more  data  regarding  the  relationship  between 
performance  time  and  experience.  AL/HRM's  productive  capacity  research  is  examining  this 
issue. 


5.2.3.  Summary 

The  data  required  to  support  SYSMOD  appear  to  be  either  available  or  obtainable 
through  a  modest  data  collection  effort.  The  conceptual  framework  appears  to  be  sound  and 
within  reach.  The  main  question  remaining  is  the  choice  of  a  simulation  or  analytic  maintenance 
manpower  model. 


5.3.  Next  Steps  in  Building  SYSMOD 

A  first  version  of  SYSMOD  could  be  started  immediately. 

•  The  BCS  Data  Base  would  contain  data  at  the  three-digit  WUC  level  for  a  few  existing 
systems,  parent  systems. 

•  Before  the  training  requirements  data  for  the  data  base  are  gathered,  the  collection 
methodology  should  be  refined. 

•  Before  the  maintenance  manpower  model  can  be  developed,  a  short  study  of  simple  simulation 
models  and  existing  analytic  models  is  needed  to  determine  which  approach  is  best. 

•  Development  of  the  MPT  LCC  cost  model,  AFMCOS,  could  begin  immediately. 

•  Limited  versions  of  first  tradeoff  models  could  be  based  on  input  modification  routines  where 
the  model  helps  anticipate  and  track  the  results  of  the  modifications. 
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GLOSSARY 


AFMCOS 

AFS 

AFSC 

AL/HRG 

AL/EIRT 

AL/HRM 

AMCOS 

ATC 

ATF 

BAQ 

BAS 

BCS 

BMT 

CHAMPUS 

CODAP 

FID 

ICNIA 

ICW 

IFF 

JQS 

KSA 

LCC 

LCOM 

LRU 

MA 

MDCS 

MPA 

MPT 

MTBF 

MTTR 

MWR 


Air  Force  Manpower  Cost  System 

Air  Force  Specialty 

Air  Force  Specialty  Code 

Armstrong  Laboratories,  Human  Resource  Dir,  Logistics  and  Human  Factors  Div 

Armstrong  Laboratories,  Human  Resource  Dir,  Training  Systems  Div 

Armstrong  Laboratories,  Human  Resource  Dir,  Manpower  &  Personnel  Div 

Army  Manpower  Cost  System 

Air  Training  Command 

Advanced  Tactical  Fighter 

Basic  Allowance  for  Quarters 

Basic  Allowance  for  Subsistence 

Baseline  Comparison  System 

Basic  Military  Training 

Civilian  Health  and  Medical  Program  of  the  Uniformed  Services 
Comprehensive  Occupational  Data  Analysis  Programs 
Field  Training  Detachment 

Integrated  Communications-Navigation-Identification  Avionics 

Integrated  Courseware 

Identification  Friend  or  Foe 

Job  Qualification  Standard 

Knowledge,  Skills,  and  Abilities 

Life-Cycle  Cost 

Logistics  Composite  Model 

Line  Replaceable  Unit 

Maintenance  Action 

Maintenance  Data  Collection  System 

Military  Personnel  Account 

Manpower,  Personnel,  and  Training 

Mean  Time  Between  Failure 

Mean  Time  To  Repair 

Morale,  Welfare,  and  Recreation 
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NRTS 

Not  Repairable  This  Station 

OJT 

On-the-Job  Training 

OS 

Occupational  Survey 

PCS 

Permanent  Change  of  Station 

PME 

Professional  Military  Education 

POI 

Plan  of  Instruction 

R&D 

Research  and  Development 

R&M 

Reliability  and  Maintainability 

RS 

Recruiting  Service 

SAAT 

Semantic  Assisted  Analysis  Technology 

SME 

Subject  Matter  Expert 

STS 

Specialty  Training  Standard 

SUMMA 

Small  Unit  Maintenance  Manpower  Analyses 

SYSMOD 

Weapon  System  Optimization  Model 

TAC 

Tactical  Air  Command 

TDS 

Training  Decisions  System 

TLD 

Task  Learning  Difficulty 

USAFOMC 

USAF  Occupational  Measurement  Center 

VHA 

Variable  Housing  Allowance 

WSAP 

Weapon  System  Acquisition  Process 

WUC 

Work  Unit  Code 

YOS 

Years  of  Service 

48 


APPENDIX  A.  F-15  LOOM  TASK  DATA 


Table  A-1  groups  the  LCOM  tasks  for  AFSC  452X1 C  for  the  F-15  by  three-digit  WUC  and 
displays  the  expected  maintenance  hours  for  each  task  resulting  from  Hying  72  sorties  per  day.  The 

three-digit  WUC  groups  are  displayed  in  order  of  the  sum  of  the  expected  workload  for  all  tasks  in 
each  group. 

Table  A-1.  F-15  Expected  Workload  by  Equipment  WUC  (Three-digit) 


TASK 

PROB 

TASK 

TIME 

CREW 

SIZE 

MAINT 

HRS/ 

SORTIE 

SORTIES 

/DAY 

MAINT 

HRS/ 

DAY 

TOTAL 

MHRS/ 

DAY/ 

WUC 

H76H00 

0.05118 

1.5 

2 

0.15353 

72  . 

11.054 

M76H00 

0.01765 

1.2 

2 

0.04235 

72 

3.049 

R76H00 

0.07500  ■ 

1.7 

2 

0.25500 

72 

18.360 

T76H00 

0.04294 

0.5 

2 

0.04294 

72 

3.092 

V76H00 

0.04897 

0.5 

2 

0.04897 

72 

3.526 

X76H00 

0.04132 

1.5 

2 

0.12397 

72 

8.926 

T76H01 

0.01471  ■ 

0.5  ■  " 

2 

0.01471 

72 

1.059 

49.07 

H76G00 

0.06609 

1.5 

2 

0.19827 

72 

14.276 

M76G00 

0.01136 

1.5 

2 

0.03409 

72 

2.455 

R76G00 

0.01345 

1.7 

2 

0.04575 

72 

3.294 

T76G00 

0.01164 

0.7 

2 

0.01629 

72 

1.173 

V76G00 

0.01391 

0.5 

2 

0.01391 

72 

1.002 

X76G00 

0.00045 

1.7 

2 

0.00155 

72 

0.111 

22.31 

H65B00 

0.01495 

1.5 

2 

0.04484 

72 

3.228 

M65B00 

0.02911 

1.2 

2 

0.06987 

72 

5.031 

R65B00 

0.01150 

1.5 

2 

0.03450 

72 

2.484 

T65B00 

0.01567 

1.0 

2 

0.03134 

72 

2.256 

V65B00 

0.00745 

0.3 

2 

0.00447 

72 

0.322 

X65B00 

0.00356 

1.3 

2 

0.00925 

72 

0.666 

13.99 

H63A00 

0.01573 

1:5 

2 

0.04719 

72 

3.398 

M63A00 

0.00731 

1.2 

2. 

0.01754 

72 

1.263 

R63A(X) 

0.01531 

1.4 

2 

0.04286 

72 

3.086 

T63A00 

0.01458 

0.3 

3  1 

0.01312 

72 

0.945 

X63A00 

0.00450 

1.3 

2  1 

0.01170 

72 

0.842 

9.53 
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PROB 

TASK 

TIME 

CREW 

SIZE 

MAINT 

HRS/ 

SORTIE 

SORTIES 

/DAY 

MAINT 

HRS/ 

DAY 

TOTAL 

MHRS/ 

DAY/ 

WUC 


H65A00 

M65A00 

R65A00 

T65A00 

V65A00 

X65A00 


0.00565 

0.03274 

0.00443 

0.01117 

0.00491 

0.00235 


H76AOO 

0.01125 

M76A00 

0.00153 

R76A00 

0.01213 

T76A00 

0.00670 

V76A00 

0.00595 

X76A00 

0.00433 

H63B00 

M63B00 

R63B00 

T63B00 

V63B00 

X63B00 


0.00236 

0.00371 

0.01176 

0.00185 

0.00285 

0.00218 


H71D00 

0.00482 

M71DOO 

0.00030 

R71D00 

0.00307 

T71D00 

0.00234 

V71DOO 

0.00146 

X71D00 

0.00058 

H71C00 

0.00142 

M71C00 

0.00029 

R71C00 

0.00227 

T71C00 

0.00057 

V71C00 

0.00114 

X71C00 

0.00114 

0.01470 

0.07203 

0.01153 

0.01564 

0.00491 

0.00610 


0.03375 

0.00366 

0.04122 

0.00938 

0.00595 

0.01298 


0.00709 

0.00890 

0.03058 

0.00260 

0.00285 

0.00567 


0.01446 

0.00077 

0.00859 

0.00327 

0.00146 

0.00152 


0.00454 

0.00074 

0.00589 

0.00136 

0.00091 

0.00341 


TASK 

PROS 

TASK 

TIME 

CREW 

SIZE 

MAINT 

HRS/ 

SORTIE 

SORTIES 

/DAY 

MAINT 

HRS/ 

DAY 

TOTAL 

MHRS/ 

DAY/ 

WUC 

H71Z00 

0.00177 

1.5 

2 

0.00530 

72 

0.382 

R71Z00 

0.00044 

1.5 

2 

0.00133 

72 

0.096 

V71Z00 

0.00133 

0.5 

2 

0.00133 

72 

0.096 

X71Z00 

0.00088 

1.4 

2 

0.00248 

72  - 

0.178 

0.75 

R76C00 

0.00103 

1.5 

2 

0.00309 

72 

0.223 

0.22 

APPENDIX  B.  ATF  LCOM  TASK  DATA 


Table  B-1  groups  the  LCOM  tasks  from  the  F-18  for  the  ATF  ICNIA  systems  by  three-digit 
WUC  and  computes  the  expected  maintenance  hours  for  each  task  resulting  from  flying  72  sorties 
per  day.  The  three-digit  WUC  groups  are  displayed  in  order  of  the  sum  of  the  expected  workload 
for  all  tasks  associated  with  the  three-digit  WUC. 


Table  B-1.  F-18  Expected  Workload  by  Equipment  WUC  (Three-digit) 


TASK 

PROB 

TASK 

TIME 

CREW 

SIZE 

MAINT 

HRS/ 

SORTIE 

SORTIES 

/DAY 

MAINT 

HRS/ 

DAY 

TOTAL 

MHRS/ 

DAY/ 

WUC 

V62X20 

0.01048 

1.90 

2 

0.03981 

72 

2.866 

H62X20 

0.00434 

0.87 

2 

0.00755 

72 

0.544 

T62X20 

0.01482 

1.78 

2 

0.05275 

72 

3.798 

M62X20 

0.00356 

1.32 

2 

0.00939 

72 

0.676 

R62X21 

0.00675 

1.65 

2 

0.02229 

72 

1.605 

R62X29 

0.00003 

2.30 

2 

0.00013 

72 

0.009 

R62X2C 

0.00006 

2.80 

2 

0.00031 

72 

0.022 

9.52 

V71D10 

0.00293 

1.90 

2 

0.01114 

72 

0.802 

M71D10 

0.00148 

2.04 

2 

0.00603 

72 

0.434 

R71D10 

0.00008 

6.93 

2 

0.00113 

72 

0.081 

H71D10 

0.00105 

2 

0.00162 

72 

0.117 

T71D10 

0.00398 

1.78 

2 

0.01418 

72 

1.021 

R71D11 

0.00048 

2.58 

2 

0.00250 

72 

0.180 

R71D12 

6.00081 

3.17 

2 

0.00512 

72 

0.368 

R71D18 

0.00008 

2 

0.00054 

72 

0.039 

3.04 

M653A0 

0.00096 

1.64 

2 

0.00316 

72 

0.227 

V653A0 

0.00316 

1.90 

2 

0.01201 

72 

0.864 

T653A0 

0.00420 

1.78 

2 

0.01496 

72 

1.077 

H653A0 

0.00104 

0.84 

2 

0.00175 

72 

0.126 

R653A3 

0.00220 

1.72 

2 

0.00756 

72 

0.544 

2.84 

H65Y10 

0.00089 

2 

0.00134 

72 

0.096 

M65Y10 

0.00129 

2 

0.00284 

72 

0.205 

T65Y10 

0.00391 

1.78 

2 

0.01391 

72 

1.001 

V65Y10 

0.00302 

1.90 

2 

0.01146 

72 

0.825 

R65Y1W 

0.00172 

2 

0.00265 

72 

0.191 

2.32 

B-1 


TASK 

PROB 

TASK 

TIME 

CREW 

SIZE 

MAINT 

HRS/ 

SORTIE 

SORTIES 

/DAY 

MAINT 

HRS/ 

DAY 

TOTAL 

MHRS/ 

DAY/ 

WUC 

T722B0 

0.00324 

0.50 

2 

0.00324 

72 

0.233 

M722B0 

0.00040 

1.21 

2 

0.00097 

72 

0.070 

V722B0 

0.00289 

0.50 

2 

0.00289 

72 

0.208 

H722B0 

0.00035 

1.14 

2 

0.00079 

72 

0.057 

R722B1 

0.00235 

2.37 

2 

0.01116 

72 

0.803 

H722CX) 

0.00163 

0.90 

2 

0.00293 

72 

0.211 

R722C1 

0.00006 

0.75 

2 

0.00009 

72 

0.006 

1.59 

T72Y10 

0.00352 

0.50 

2 

0.00352 

72 

0.254 

M72Y10 

0.00117 

2.25 

2 

0.00524 

72 

0.378 

V72Y10 

0.00305 

0.50 

2 

0.00305 

72 

0.220 

R72Y1A 

0.00003 

1.20 

2 

0.00007 

72 

0.005 

R72Y1F 

0.00017 

3.05 

2 

0.00101 

72 

0.073 

R72Y1H 

0.00003 

1.80 

2 

0.00010 

72 

0.007 

R72Y1V 

0.00006 

2.45 

2 

0.00027 

72 

0.019 

R72Y1W 

0.00161 

1.44 

2 

0.00463 

72 

0.333 

1.29 

T621K0 

0.00171 

1.78 

2 

0.00608 

72 

0.438 

R621K0  - 

-  0.00023 

1.16 

2 

0.00054 

72 

0.039 

V621K0 

0.00087 

1.90 

2 

0.00330 

72 

0.238 

M621K0 

0.00064 

2.11 

2 

0.00268 

72 

0.193 

H621K0 

0.00084 

0.80 

2 

0.00135 

72 

0.097 

1.01 

T66140 

0.00141 

1.78 

2 

0.00503 

72 

0.362 

M66140 

0.00003 

0.50 

2 

0.00003 

72 

0.002 

V66140 

0.00141 

1.90 

2 

0.00537 

72 

0.387 

0.75 

A74000 

0.00202 

2.49 

2 

0.01007 

72 

0.725 

0.73 

R64X11 

0.00259 

1.44 

2 

0.00747 

72 

0.538 

R64X12 

0.00006 

1.95 

2 

0.00022 

72 

0.016 

0.55 

H729D0 

0.00040 

0.97 

2 

0.00078 

72 

0.056 

R729D0 

0.00003 

0.70 

2 

0.00004 

72 

0.003 

R729D1 

0.00026 

3.57 

2 

0.00183 

72 

0.132 

R729D9 

0.00003 

1.00 

2  - 

0.00006 

72 

0.004 

0.20 

V72RA0 

0.00043 

0.50 

2 

0.00043 

72 

0.031 

M72RA0 

0.00032 

2.82 

2 

0.00180 

72 

0.130 

T72RA0 

0.00045 

0.50 

2 

0.00045 

72 

0.033 

0.19 
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APPENDIX  C.  CORRELATING  LCOM  TASKS  WITH  OS  TASKS 


C4>  Introduction 

LCOM  tasks  are  used  by  the  manpower  community  as  a  structure  for  organizing  manpower 
requirements  and  cost  data.  For  example,  maintenance  time  and  crew  size  are  identified  with  LCOM 
tasks  to  account  for  manpower  needs.  The  study  attempted  to  use  OS  tasks  similarly  as  a  structure  for 
organizing  training  requirements  and  cost  data.  The  STS  is  based  upon  the  complete  set  of  OS  tasks  for 
an  AFS  and  is  the  starting  point  for  developing  POIs  for  formal  training  courses  and  for  the  Air  Force 
JQS  or  OJT  record.  Then,  to  link  manpower  requirements  or  cost  data  wiih  training  requirements  or  cost 
data,  the  study  had  to  correlate  LCOM  tasks  with  OS  tasks.  For  instance,  the  study  tried  to  tie  training 
hours  to  LCOM  tasks  by  first  linking  training  hours  to  OS  tasks  and  then  mapping  OS  tasks  to  LCOM 
tasks  to  assign  the  hours  to  LCOM  tasks. 

C.2.  Issues  Affecting  Correlation 

In  general,  LCOM  tasks  are  identified  by  two-component  codes.  The  first  component  is  a 
character  which  identifies  the  maintenance  action  (MA)  to  be  performed,  e.g.  T  indicates  Troubleshoot. 
The  second  component  is  the  WUC,  a  5-position  alphanumeric  code  which  identifies  the  equipment 
requiring  the  maintenance  action,  e.g.  63 AGO  -  UHF  Communications  Set.^'  Thus  each  LCOM  task  can 
be  cleanly  translated  to  a  MAAVUC  combination.  OS  tasks  are  identified  by  a  letter  followed  by  a 
sequence  number.  The  letter  groups  tasks  into  general  categories  such  as: 

Performing  General  Avionics  Systems  Maintenance  Tasks,  and 
Maintaining  Comm/Nav/Pen  Aids  Systems 
The  two  nomenclature  systems  do  not  lend  themselves  to  direct  correlation. 

However,  the  two  systems  can  be  roughly  correlated  through  task  descriptions.  OS  task 
descriptions  consist  of  an  action  verb  and  an  object  of  the  action.  OS  action  verbs  and  LCOM 
maintenance  actions  can  be  mapped  as  shown  in  Table  C-1  (Metrica,  1991).  LCOM  WUCs  link  directly 
to  object  descriptions  in  the  Aircraft  Maintenance  Work  Unit  Code  Manual  (commonly  referred  to  as  the 
Dash  06).  However,  correlating  object  descriptions  from  OS  tasks  with  WUC  descriptions  in  the  Dash 
06  is  a  complicated  process  that  ultimately  affects  the  correlation  of  the  two  task  structures.  Because  the 


WUCs  are  established  and  maintained  independently  of  either  LCOM  or  OS  type  data  bases.  The  codes  fill 
2  to  5  digits  of  the  5-position  task  identifier,  where  a  2-digit  code  indicates  one  of  the  major  systems  of  the  aircraft, 
e.g.  63000  UHF  Communications  System;  a  3-digit  code  indicates  a  set,  e.g.  63A00  UHF  Communications  Set, 
which  is  part  of  the  parent  2-digit  WUC  system;  a  4-digit  code  indicates  an  LRU,  e.g.  63AA0  UHF 
Receiver/Transmitter,  which  is  part  of  the  parent  3-digit  WUC  set;  and  the  5-digit  code  indicates  an  assembly,  e.g. 
63AAA  Chassis  Assembly,  which  is  part  of  the  parent  4-digit  unit.  WUCs  and  associated  hardware  are  listed  in  the 
aircraft’s  B-4  Master  File  and  in  the  Aircraft  Maintenance  Work  Unit  Code  Manual. 
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two  descriptions  use  neither  the  same  terminology  nor  the  same  level  of  aggregation,  these  differences 
lead  to  problems  in  matching  tasks. 

Table  C-1.  Correlation  of  LCOM  Actions  and  OS  Verbs 


LCOM  ACTION 

TAKEN 

OS  VERB 

Verify 

Operationally  check 

Troubleshoot 

Troubleshoot 

Unscheduled  remove 
and  replace 

•  Disassemble 

•  Remove  or  install 

Repair  in  place/Minor 
maintenance 

•  Adjust 

•  Calibrate 

•  Repair/replace  minor  parts 

•  Clean 

•  Corrosion  Repair 

Cannot  duplicate 

Flightline  check 

Remove  and  replace  to 
facilitate  other 

maintenance 

For  example,  the  AFS  452X1C  OS  task  list  includes: 

G278  Perform  safety  wiring 

Safety  wiring  cannot  be  directly  related  to  any  one  piece  of  equipment  described  in  the  Dash  06.  Further, 
differences  in  aggregation  lead  to  multiple  matches.  For  example,  the  Semantic- Assisted  Analysis 
Technology  (SAAT)  software  developed  by  AL/HRM  mapped  the  AFS  326X8C  OS  task  -  Isolate 
malfunction  to  UHF  antenna  -  to  the  following  WUCs  and  object  descriptions  from  the  Dash  06^: 
63000  UHF  Communication 

63B00  Communication  Set  UHF 

63BD0  Selector  Antenna 
63BEO  Antenna  dual  band  upper 


^  The  SAAT  methodology  is  described  in  (Metrica,  1991).  AFSC  326X8C  was  converted  to  AFSC  452X2C 
in  1988. 
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63BF0  Antenna  dual  band  lower 

Thus  the  OS  task  maps  to  five  trouble  shooting  tasks,  one  for  each  WUC. 

A  second  issue  affecting  the  correlation  of  the  two  task  structures  is  more  fundamental.  The 
maintenance  and  training  communities  have  different  perspectives  on  maintenance  activities.  The 
maintenance  community  takes  a  functional  approach  that  is  structured  along  WUC  lines.  It  is  concerned 
with  the  day-to-day  maintenance  of  equipment,  and  the  equipment  fits  neatly  into  the  WUC  structure. 
The  training  community  is  concerned  about  all  tasks  performed  by  members  of  the  AFS,  including  non¬ 
maintenance  tasks,  and  enabling  members  of  the  AFS  to  perform  the  tasks.  Thus  the  training  community 
takes  a  skills  and  knowledge  approach  that  may  cut  across  WUC  boundaries  or  even  be  outside  the  WUC 
domain.  The  two  communities  slice  maintenance  activities  into  tasks  differently.  Sometimes  there  is  not 
a  one-to-one  correspondence  between  the  two  slices. 

C.3.  Results 

Since  SAAT  analysis  had  not  been  done  on  the  452X1C  data,  we  correlated  LCOM  and  OS  tasks 
by  hand.  In  doing  so,  we  faced  the  issues  identified  above.  We  found  LCOM  tasks  that  had  no 
corresponding  OS  task  and  others  that  had  several.  Conversely,  we  found  OS  tasks  that  had  no 
corresponding  LCOM  task  and  others  that  had  several.  To  improve  our  mapping,  we  had  our  results 
reviewed  by  an  Air  Force  SME.  The  review  eliminated  problems  we  had  relating  object  descriptions 
from  the  two  systems,  but  it  did  not  eliminate  problems  due  to  different  levels  of  aggregation  and  the 
fundamental  differences  in  orientation  of  the  maintenance  and  the  training  communities. 

The  correlation  between  LCOM  and  OS  tasks  for  AFS  452X1C  is  attached.  (See  Table  C-2.) 
LCOM  tasks  are  listed  on  the  left-hand  side  and  OS  tasks  along  the  right-hand  side.  WUC  descriptions 
from  the  Dash  06  are  in  the  middle. 

C.4.  Comments 

The  attached  mapping  and  a  similar  one  for  the  F-16  (452X2C)  were  the  starting  point  for  a 
number  of  analyses  conducted  in  the  study.  The  F-16  mapping  was  used  to  relate  learning  difficulty  for 
OS  tasks  to  SUMMA  (LCOM)  tasks  in  the  regression  studies  on  estimating  task  performance  times  for 
a  secondary  AFS.  A  proration  scheme  was  used  to  handle  the  cases  where  several  OS  tasks  mapped  to 
a  single  LCOM  task. 

\ 

The  study  tried  to  use  the  F-15  mapping  to  link  training  hours  to  LCOM  tasks  in  order  to  define 
the  requirements  for  the  cross  training  courses.  A  combination  of  problems  led  the  study  to  modify  the 
planned  approach.  First,  the  mapping  of  OS  tasks  to  the  POIs  for  the  initial  skills  and  FTD  courses  from 
the  Training  Extract  (USAFOMC,  1990a)  left  some  OS  tasks  unaccounted  for  and  had  some  POI  topics 
not  linked  to  any  OS  tasks.  These  problems,  compounded  by  the  mapping  problems  between  OS  and 
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LCOM  tasks,  produced  weak  relationships  between  course  hours  and  LCOM  tasks.  To  minimize  these 
problems,  the  study  grouped  tasks  into  sets  that  included  all  tasks  associated  with  a  three-digit  WUC. 
Thus  all  OS  tasks  or  all  LCOM  tasks  concerned  with  the  three-digit  WUC  were  considered  together.  The 
study  then  had  an  SME  map  POI  topics  directly  to  three-digit  WUC  to  identify  the  training  requirements 
for  each  set  of  tasks.  This  eliminated  some  of  the  mapping  problems.  (See  Section  3.2). 
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APPENDIX  D.  COMPUTATION  OF  MEASURES  OF  TASK 
CO-PERFORMANCE 


D.l.  Description  of  Co-Performance  Computations 

In  the  overlap  program  (OVRLAP)  of  the  Comprehensive  Occupational  Data  Analysis  Programs 
(CODAP),  task  co-performance  is  defined  as  a  measure  of  the  similarity  of  pairs  of  task  profiles  across 
all  the  people  in  an  occupational  survey  sample  (Phalen,  1989).  Since  previous  studies  have  found  that 
co-performed  tasks  tend  to  require  similar  skills,  knowledge,  and  abilities,  this  study  used  co-performance 
to  identify  sets  of  tasks  for  cross  training  in  order  to  limit  the  costs  of  the  cross  training  (Perrin,  1988 
and  Lamb,  1989). 

CODAP  Co-Performance  Clustering:  Co-performance  clustering  is  a  multi-step  process  carried 
out  by  CODAP.  The  steps  are  described  through  an  illustrative  example  in  a  manner  similar  to  that  used 
for  the  co-performance  measure  in  the  Training  Decisions  System  (Lamb,  1989). 

Step  1:  Figure  D-1  is  an  example  of  a  portion  of  a  job  inventory  for  AFS  452X1C  (F-15 
Communications,  Navigation,  and  Penetration  Aids  Systems).  Each  person  who  completes  the  survey 
indicates  the  amount  of  time  he  or  she  spends  on  each  task  using  a  nine-point  scale  ranging  from  "very 
small  amount"  to  "very  large  amount".  These  raw  data  are  used  to  compute  co-performance. 


CHECK 

Time  spent  present  job 

IF 

DONE 

NOW 

RATE 

1 .  Very  small  amount. 

2.  Much  below  avg. 

3.  Below  avg. 

4.  Slightly  below  avg. 

5.  About  avg. 

6.  Slightly  above  avg. 

7.  Above  avg. 

8.  Much  above  avg. 

9.  Very  large  amount. 

A.  Isolate  malfunctions  to  secure 
voice  crypto  equipment 

123456789 

B.  Perform  ops  checkout/BIT  of  UHF 
comm/audio  signal  systems 

123456789 

C.  Remove  or  install  secure  voice 
crypto  equipment  LRUs 

123456789 

D.  Remove  or  install  UHF  comm 
and  audio  signal  systems 

123456789 

Figure  D-1.  Format  of  Job  Inventory 
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Step  2:  A  case-oriented  data 
file  is  developed  that  contains  the 
responses  from  all  the  people  (cases) 
surveyed.  See  Figure  D-2  for  a 
sample  of  ratings  from  an  inventory. 


Case  1 

Case  2 

Case  3 

Case  4 

Task  A 

5 

7 

7 

7 

Task  B 

5 

6 

7 

4 

Task  C 

5 

3 

5 

7 

Task  D 

5 

4 

6 

7 

Total 

20 

20 

25 

25 

Figure  D-2.  Case-oriented  Data  File  from  Job  Inventory 


Step  3:  CODAP  converts  the 
entries  in  the  case-oriented  data  file  to 
percent  time  for  each  case.  See 
Figure  D-3.  For  example.  Case  1 
spends  25%  of  his  or  her  time 
performing  Task  A.  The  total 
percent  time  is  100%  for  each  case. 


Case  1 

Case  2 

Case  3 

Case  4 

Task  A 

25 

35 

28 

28 

Task  B 

25 

30 

28 

18 

Task  C 

25 

15 

20 

28 

Task  D 

25 

20 

24 

28 

Total 

100 

100 

100 

100 

Figure  D-3.  Case-oriented  Data  File  Converted  to 


Percentages 


Step  4:  To  conform 
CODAP  requirements,  the  data 
must  be  transposed  as  shown 
Figure  D-4. 


Task  A 

Task  B 

Task  C 

Task  D 

Case  1 

25 

25 

25 

25 

Case  2 

35 

30 

15 

20 

Case  3 

28 

28 

20 

24 

Case  4 

28 

18 

28 

28 

Total 

116 

101 

88 

97 

Figure  D-4.  Task-oriented  Data  File 
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Step  5:  CODAP  converts  the 
entries  in  the  task-oriented  data  file  to 
percentages  for  each  task  due  to  each 
case.  See  Figure  D-5.  For  example, 
22%  of  the  time  spent  on  Task  A  by 
these  four  cases  is  spent  by  Case  1. 
The  sura  of  the  percentage  values  is 
100%  for  each  task. 


r 

Task  A 

TaskB 

TaskC 

Task  D 

Case  1 

22 

25 

28 

26 

Case  2 

30 

31 

17 

21 

Case  3 

24 

28 

23 

25 

Case  4 

24 

16 

32 

28 

Total 

100 

100 

100 

100 

Figure  D-5.  Task-oriented  Data  File  as  Percentage  by  Task 


Step  6:  The  measure  of  co-performance  between  each  pair  of  tasks  is  computed  as  the  sum  of 
the  minimum  entry  for  each  case  i  as  follows: 

n 

coperj^iA^) 

i=l 


where 

A;  =  task  A  percentage  for  case  i 
Bj  =  task  B  percentage  for  case  i 
n  =  number  of  cases 


Figure  D-6  gives  the  co-performance 
measure  for  all  pairwise  combinations 
of  tasks. 

For  example, 

copeT^„{A^)  =22+30+24+16=92 


Task  A 

Task  B 

Task  C 

Task  D 

Task  A 

100 

92 

86 

91 

Task  B 

92 

100 

81 

87 

Task  C 

86 

81 

100 

94 

Task  D 

91 

87 

94 

100 

Figure  D-6.  Co-performance  Matrix  for  Four  Groups 


Step  7:  Tasks  C  and  D  are  identified  as  the  first  tasks  to  be  clustered  because  they  have  the 
highest  co-performance  measure  (94%).  When  CODAP  merges  tasks  C  and  D,  it  computes  a  measure 
of  group  similarity  called  the  co-performance  "within"  the  new  group.  This  measure  is  calculated  as  the 
average  co-performance  of  each  pairwise  combination  of  tasks  in  the  group  including  each  task  with  itself. 
For  the  group  CD,  this  computation  is  as  follows: 


cop€Tf^„(CD)= 


coper^iC,Q+coperf^iCfi)+cope7^^iD,Q+coperf^(D^) 

4 
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A  more  meaningful  measure  of  co-performance  "within"  would  be  one  which  excludes  the  overlap 
of  each  task  with  itself  (i.e. ,  100  %).  Such  a  measure,  which  might  be  called  co-performance  within  and 
be  designated  coperf^/„,  can  be  calculated  for  group  CD  using  the  following  equation: 


copejf^J.CD)= 


coper^„{Cjy) +coperff^{D,C) 


Thus, 


However,  since  CODAP  does  not  provide  this  calculation,  the  same  result  can  be  accomplished  by  using 
the  copetf„/„  from  CODAP  in  the  following  equation: 

n[coperf^„{..:)ym 

copeif^ni...) = - ^ - 

1%  1 


where  n  is  the  number  of  tasks  in  the  group.  Thus, 


/.rm  2(97) -100  q. 
coperf,i„iCD)=^^ - =94 


Once  CODAP  computes  the  coperf^JCD),  it  then  collapses  rows  C  and  D  and  columns  C  and 
D  of  the  overlap  matrix  into  a  single  row  and  column,  thereby  reducing  the  size  of  the  matrix.  This  is 
done  by  computing  a  measure  of  the  average  co-performance  "between"  each  task  in  the  new  group  and 
each  task  in  each  of  the  remaining  groups.  For  example,  the  co-performance  between  group  CD  and 
group  A  is  computed  as  follows: 


coperft^iCD,A)= 


coper^iC^VcoperCnW) 


coperfj^„(CDA>  =88.5 
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The  co-performance  between  groups  CD  and 
B  (i.e.,  copeif,^(CD,B))  is  computed 
analogously.  Figure  D-7  gives  the  co- 
peiformance  measures  for  the  three  remaining 
groups. 


I-.: 

Task  A 

Task  B 

Task  CD 

Task  A 

100 

92 

88.5 

Task  B 

92 

100 

84 

Task  CD 

88.5 

84 

97 

Figure  D-7.  < 

Do-performance  Matrix  after  Combining 

Groups  C  and  D 


Step  8:  Tasks  A  and  B  are  identified  as  the  next  tasks  to  be  merged  because  they  now  have  the 
highest  co-performance  measure  (92%).  The  co-performance  within  the  new  group  is  computed  as 
follows: 


thus, 


and. 


cope?f^„iAB)= 


nlcoperf^JAB)]  -100  _  2(96)-100 
ra-1  1 


=92 


The  co-performance  between  groups  AB  and  CD  is  as  follows: 


copetfi^iAB,CD)= 


4 


thus. 


copeTf^^„(AB,CD)= 


(86+91+81+87) 

4 


=86.25 


Figure  D-8  gives  the  co-performance  measures  for  the  two  groups  remaining. 
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Step  9:  The  next  most  similar  pair  is  now  the 
only  remaining  pair,  groups  AB  and  CD.  The  co¬ 
performance  within  the  new  group  ABCD  is  as 
follows: 

copeTf^„(ABCD)=^J^  ^  coper^ijji) 

lOjM  k=A 


Task  AB 

Task  CD 

Task  AB 

96 

86.25 

Task  CD 

86.25 

97 

Figure  D-8.  Co-performance  Matrix  for  Two 
Groups 


thus, 

=91.38 


and. 


Figure  D-9  gives  the  co-performance  within  the  single 
group  containing  all  four  tasks. 


1 

Task  ABCD 

1  Task  ABCD 

91.38 

Figure  D-9.  Co-perfonnance  Measure  for 
Single  Group 
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Another  value  of  interest  is  the  average  co-performance  of  each  task  in  a  group  with  every  other 
task  in  the  group,  computed  by  the  TASSET  program  in  CODAP.  For  this  final  group  of  four  tasks  the 
average  co-performance  for  task  A  is 


copeTf^(A)= 


coper/^iA^)+cope7j^iA,Q+copeij^ 

3 


thus. 


coperf^(A)=^^^^^  =89.67 


Similarly, 


copeTf^{B)=^!t^^=Z6.61 

The  range  of  these  values  gives  a  sense  of  the  compactness  of  the  group  of  tasks,  86.67%  to 
90.67  %  in  this  case.  Also,  if  a  single  task  were  needed  to  represent  the  group,  one  choice  would  be  the 
task  with  the  highest  average  co-performance  with  all  other  tasks  in  the  group  (i.e.,  task  D). 
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D.2.  Co-Performance  as  Applied  to  F-15  COMM/NAV/PEN  Aids  Tasks 


The  study  applied  the  hierarchical 
clustering  techniques  of  the  CODAP  routines 
to  F-15  COMM/NAV/PEN  AIDS  tasks 
performed  by  AFS  452X1C^®.  CODAP 
performed  the  initial  clustering  and  displayed 
the  results  as  a  tree  diagram  composed  of  data 
blocks  with  the  format  shown  in  Figure  D-10. 

The  branches  of  the  diagram  traced  the  paths 
of  individual  tasks  as  they  were  combined  into 
clusters  and  of  the  clusters  as  they  were  combined  into  larger  clusters.  (See  Figure  D-ll.y° 


Figure  D-10.  CODAP  Cluster  Hierarchy  Data  Block 


100% 


ao% 


Figure  D-11.  Module  A  Tasks  Displayed  by  Co-Performance  Percentages 

Using  the  tree  diagram,  we  examined  the  CODAP  ouq)ut  to  evaluate  clusters  and  group  tasks  to 
be  handled  together  in  cross  training  modules.  We  began  by  grouping  task  clusters  based  on  a  rough 
visual  analysis  of  the  tree  diagram.  Although  this  process  tended  to  separate  clusters  into  distinct  groups 


CODAP  was  executed  on  69  Occupational  Survey  (OS)  tasks  from  CODAP  study  9598  (conducted  as  a  part 
of  the  occupational  survey  of  the  F-15  avionics  systems  career  ladder  in  1990),  tasks  with  CODAP  T-Path  numbers 
116  through  184,  for  people  with  48  months  of  service  or  less.  This  range  of  tasks  was  chosen  to  include  all 
452X1C  COMM/NAV/IFF  tasks  (T-Path  numbers  124-161  and  176-184),  identified  in  OS  data  as  J  tasks.  Tasks 
with  T-Path  numbers  162-175' involve  cables,  wiring,  and  wave  guides  and  are  identified  in  OS  data  as  G  tasks. 
Miscellaneous  other  tasks  were  included  in  the  cluster  analysis  because  of  their  co-performance  with  the  above  tasks; 
they  are  T-Path  numbers  116-123  which  include  1  E  task,  4  F  tasks,  and  3  G  tasks. 

“  The  tasks  identified  in  Figure  D-11  are  connected  by  lines  indicating  the  co-performance  between  tasks  or 
groups  when  they  merge.  The  positions  and  sizes  of  task  blocks  have  no  additional  meaning. 
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by  WUC,  it  did  not  necessarily  aggregate  clusters  by  WUC.  This  first  cut  associated  the  69  tasks  into 
12  groups  ranging  in  size  from  1  task  to  14  tasks.  See  Table  D-1. 

We  revised  the  groups  following  a  closer  look  at  the  tasks  in  each  group  and  their  associated 
WUCs.  The  final  groups  resulted  from  application  of  the  following  guidelines. 

•  In  general,  if  groups  containing  multiple  WUCs  could  be  merged  without  increasing  the 
variety  of  WUCs  in  the  final  group,  they  were  merged.  For  example,  three  of  the  original 
groups  (A,B,C)  containing  tasks  for  WUC  63,  63  and  65,  and  63  respectively  were  combined 
to  make  the  revised  A  group. 

•  Groups  with  common  two-digit  WUCs  were  not  merged  if  combining  the  groups  broadened 
the  range  of  included  three-digit  WUCs.  For  example.  Group  H  (originally  Group  J)  contained 
only  76F  tasks  and  was  the  only  group  to  contain  76F  tasks.  It  was  not  merged  with  other  76 
groups. 

•  Groups  with  common  WUCs  were  not  merged  if  combining  the  groups  caused  a  relatively 
large  change  in  co-performance.  For  example,  group  A  contains  WUC  63  and  65  tasks,  and 
group  D  contains  WUC  63  tasks.  The  values  of  co-performance  within  those  groups  were 
83.2%  and  82.6%  respectively;  however,  if  the  groups  were  combined,  the  co-performance 
between  the  merging  groups  was  66.1%.  Therefore,  we  kept  the  two  groups  separate.  See  Table 
D-2  for  measures  of  co-performance  between  and  co-performance  within  groups. 

The  revised  groupings  are  displayed  as  blocks  in  Figure  D-12.  The  height  of  each  block 
represents  the  range  of  the  average  co-performance  measure  of  the  tasks  in  the  group  with  other  tasks  in 
the  group  (from  TASSET).  The  width  of  each  box  represents  the  number  of  tasks  included  in  the  group. 
The  vertical  position  of  each  horizontal  line  connecting  two  groups  indicates  the  co-performance  between 
the  groups  when  the  groups  merge.  For  example.  Group  A  contains  13  tasks  having  average  co¬ 
performance  measures  from  TASSET  ranging  from  83.2%  to  87.8%,  while  Group  D  contains  4  tasks 
with  average  co-performance  measures  ranging  from  82.6%  to  84.4%.  There  are  7  major  groups,  A 
through  E,  G,  and  H,  containing  firom  3  to  13  tasks  each,  specifically  on  COMM/NAV/PEN  AIDS 
equipment.  An  eighth  major  group  (F)  contains  14  cable,  wiring,  and  wave  guide  tasks.  Two  additional 
groups,  X  and  Z,  each  with  four  tasks,  contain  safety  tasks  and  miscellaneous  other  tasks. 
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Figure  D-12.  F>15  OS  Tasks  Aggr^ated  by  Co-Performance 


Average  co-performance  measures  from  TASSET  for  the  tasks  in  each  group  are  given  in  Table 
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Table  D-1.  Association  of  Tasks  by  Co-Performance 


The  Final  Group  column  indicates  the  revised  group  assignments  of  OS  tasks  based  upon  WUCs.  The  Original 
Group  column  indicates  the  initial  assignment  of  tasks  to  groups  based  on  a  rough  visual  examination  of  the  cluster 


diagram  output  by  CODAP. 

CODAP 


FINAL 

ORIG 

T-PATH 

os  TASK 

GROUP 

GROUP 

NUMBER 

wuc 

NUMBER 

A 

A 

125 

65A,65B 

J407 

A 

126 

65A,65B 

J410 

A 

127 

65A,65B 

J412 

A 

128 

65A,65B 

J436 

A 

129 

65A,65B 

J439 

A 

130 

65A,65B 

J441 

B 

131 

63 

J420 

B 

132 

63 

J432 

B 

133 

63 

J449 

B 

134 

65 

J424 

B 

135 

65 

J426 

B 

136 

65 

J427 

C 

124 

65 

J405 

B 

D 

137 

71D,71Z 

J415 

P 

138 

71D,71Z 

J428 

D 

139 

71D,71Z 

J443 

C 

E 

140 

71B 

J411 

, 

E 

141 

76C 

J414 

E 

142 

71C 

J413 

E 

143 

71B 

J425 

E 

144 

71C 

J433 

E 

145 

76C 

J421 

E 

146 

71B 

J437 

OS  TASK 

DESCRIPTION 

* 

ISOLATE  MALFUNCTIONS  TO  MODE  4  CRYPTO 
EQUIPMENT 

ISOLATE  MALFUNCTIONS  WITHIN  AIR-TO-AIR  IFF 
INTERROGATOR  (AAI)  SYSTEMS 
ISOLATE  MALFUNCTIONS  WITHIN  IDENTMCATION 
FRIEND  OR  FOE  (IFF)  SYSTEMS 
REMOVE  OR  INSTALL  AAI  SYSTEM  LINE 
REPLACEABLE  UNITS  (LRU) 

REMOVE  OR  INSTALL  IFF  SYSTEM  LRUS 

REMOVE  OR  INSTALL  MODE  4  CRYPTO  EQUIPMENT 

LRUS 

ISOLATE  MALFUNCTIONS  WITHIN  ULTRA  HIGH 
FREQUENCY  (UHF)  COMMUNICATION  AND  AUDIO 
SIGNAL  SYSTEMS 

PERFORM  OPERATIONAL  CHECKOUT  AND  BIT  OF 
UHF  COMMUNICATION  AND  AUDIO  SIGNAL 
SYSTEMS 

REMOVE  OR  INSTALL  UHF  COMMUNICATION  AND 
AUDIO  SIGNAL  SYSTEM  LRUS 
PERFORM  OPERATIONAL  CHECKOUT  AND  BIT  OF 
AAI  SYSTEMS 

PERFORM  OPERATIONAL  CHECKOUT  AND  BIT  OF  IFF 
SYSTEMS 

PERFORM  OPERATIONAL  CHECKOUT  AND  BIT  OF 
MODE  4  CRYPTO  EQUIPMENT 
CODE  MODE  4  CRYPTO  EQUIPMENT 

ISOLATE  MALFUNCTIONS  WITHIN  TACTICAL  AIR 
NAVIGATION  (TACAN)  SYSTEMS 
PERFORM  OPERATIONAL  CHECKOUT  AND  BIT  OF 
TACAN  SYSTEMS 

REMOVE  OR  INSTALL  TACAN  SYSTEM  LRUS 

ISOLATE  MALFUNCTIONS  WITHIN  AUTOMATIC 
DIRECTION  FINDER  (ADF)  SYSTEMS 
ISOLATE  MALFUNCTIONS  WITHIN  INTERFERENCE 
BLANKER  SYSTEMS  (IBS) 

ISOLATE  MALFUNCTIONS  WITHIN  INSTRUMENT 
LANDING  SYSTEMS  (TLS) 

PERFORM  OPERATIONAL  CHECKOUT  AND  BIT  OF 
ADF  SYSTEMS 

PERFORM  OPERATIONAL  CHECKOUT  OF  ILSS 

PERFORM  BIT  OF  IBSS 

REMOVE  OR  INSTALL  ADF  SYSTEM  LRUS 
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CODAP 

FINAL 

ORIG 

T-PATH 

os  TASK 

GROUP 

GROUP 

NUMBER 

wuc 

NUMBER 

c 

E 

-147 

76C 

J438 

E 

148 

71C 

J440 

D 

F 

149 

63 

J406 

F 

150 

63 

J408 

F 

151 

63 

J434 

F 

152 

63 

J442 

E 

G 

153 

76A 

J409 

G 

154 

76A 

J431 

G 

155 

76A 

J448 

G 

156 

76H 

J419 

G 

157 

76H 

J430 

G 

158 

76H 

J447 

G 

159 

76G 

J417 

G 

160 

76G 

J446 

G 

161 

76G 

J429 

F 

H 

162 

G265 

H 

163 

G267 

H 

164 

G269 

H 

165 

G271 

H 

166 

G266 

H 

167 

G272 

H 

168 

G273 

H 

169 

G275 

H 

170 

G288 

H 

171 

G280 

H 

172 

G281 

H 

173 

G284 

H 

174 

G289. 

H 

175 

G287 

OS  TASK 
DESCRIPTION 

REMOVE  OR  INSTALL  IBS  LRUS 
REMOVE  OR  INSTALL  ILS  LRUS 

CODE  SECURE  VOICE  CRYPTO  EQUIPMENT 
ISOLATE  MALFUNCTIONS  TO  SECURE  VOICE 
CRYPTO  EQUIPMENT 

PERFORM  OPERATIONAL  CHECKOUT  OF  SECURE 
VOICE  CRYPTO  EQUIPMENT 
REMOVE  OR  INSTALL  SECURE  VOICE  CRYPTO 
EQUIPMENT  LRUS 

ISOLATE  MALFUNCTIONS  TO  TACTICAL 
ELECTRONIC  WARFARE  SYSTEM  (TEWS)  RADAR 
WARNING  RECEIVERS  (RWR) 

PERFORM  OPERATIONAL  CHECKOUT  AND  BIT  OF 
TEWS  RWRS 

REMOVE  OR  INSTALL  TEWS  RWR  LRUS 
ISOLATE  MALFUNCTIONS  WITHIN  TEWS  INTERNAL 
COUNTERMEASURES  SYSTEMS  aCMS) 

PERFORM  OPERATIONAL  CHECKOUT  AND  BIT  OF 
TEWS  ICMSS 

REMOVE  OR  INSTALL  TEWS  ICMS  LRUS 
ISOLATE  MALFUNCTIONS  WITHIN  TEWS 
ELECTRONIC  WARFARE  WARNING  SYSTEMS  (EWWS) 
REMOVE  OR  INSTALL  TEWS  EWWS  LRUS 
PERFORM  OPERATIONAL  CHECKOUT  AND  BIT  OF 
TEWS  EWWSS 

INSPECT  AIRCRAFT  WIRING 

INSPECT  COAXIAL  CABLES  AND  CONNECTORS 

INSPECT  MULTIPIN  CONNECTORS 

INSPECT  WAVEGUIDES 

INSPECT  CHAFING  PROBLEM  AREAS 

ISOLATE  MALFUNCTIONS  WITHIN  AIRCRAFT  WIRING 

ISOLATE  MALFUNCTIONS  WITHIN  COAXIAL  CABLES 
AND  CONNECTORS 

ISOLATE  MALFUNCTIONS  WITHIN  MULTIPIN 

CONNECTORS 

REPAIR  AIRCRAFT  WIRING 

REMOVE  OR  INSTALL  COAXIAL  CABLES 

REMOVE  OR  INSTALL  COAXIAL  CONNECTORS 

REMOVE  OR  INSTALL  MULTIPIN  CONNECTORS 

(CANNON  PLUG) 

REPAIR  CHAFED  AREAS 
REMOVE  OR  INSTALL  WAVEGUIDES 
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CODAP 


FINAL 

ORIG 

T-PATH 

os  TASK 

GROUP 

GROUP 

NUMBER 

wuc 

NUMBER 

G 

I 

176 

76_ 

J416 

I 

177 

76_ 

J422 

I 

178 

J444 

I 

179 

76H 

J435 

H 

J 

180 

76F 

J418 

J 

181 

76F 

J445 

J 

182 

76F 

J450 

J 

183 

76F 

J451 

J 

184 

76F 

J423 

X 

X 

120 

F186 

121 

FI  87 

122 

Gill 

123 

G278 

Z 

z 

116 

E130 

117 

F232 

118 

F261 

119 

G263 

OS  TASK 
DESCRIPTION 

ISOLATE  MALFUNCTIONS  WITHIN  TEWS 
COUNTERMEASURES  DISPENSER  SYSTEMS 
PERFORM  BIT  OF  TEWS  COUNTERMEASURES 
DISPENSER  SYSTEMS 

REMOVE  OR  INSTALL  TEWS  COUNTERMEASURES 
DISPENSER  SYSTEM  LRUS 

PERFORM  PRESSURIZATION  TESTS  OF  TEWS  ICMSS 

ISOLATE  MALFUNCTIONS  WITHIN  TEWS  EXTERNAL 
COUNTERMEASURES  SYSTEMS  (ECMS)(PODS) 
REMOVE  OR  INSTALL  TEWS  ECMS  (PODS)  LRUS 
TRANSPORT  TEWS  ECMS  (PODS) 

UPLOAD  OR  DOWNLOAD  TEWS  ECMS  (PODS) 
PERFORM  BIT  OF  TEWS  ECMSS  (PODS) 

CONNECT  OR  DISCONNECT  AIRCRAFT  EXTERNAL 
COOLING  AIR  UNITS 

CONNECT  OR  DISCONNECT  AIRCRAFT  EXTERNAL 
POWER 

PERFORM  AIRCRAFT  SAFE  FOR  MAINTENANCE 
CHECKS 

PERFORM  SAFETY  WIRING 

INITIATE  AFTO  FORMS  350  (REPARABLE  ITEM 
PROCESSING  TAG) 

POSITION  OR  REMOVE  AIRCRAFT  CHOCKS  OR 
SAFETY  PINS 

WALK  WINGS  OR  TAILS  DURING  AIRCRAFT  TOWING 
OPERATIONS 

ANALYZE  AVIONICS  STATUS  PANEL  (ASP)  LATCH 
DATA 


Table  D-2.  F-15  COMM/NAV/PEN  AIDS  Co-Performance 


TASK  RANGE 

CO-PERF 

BETWEEN 

MERGING 

CO-PERF 

WITHIN” 

CLUSTERING 

#  TASKS 

IN 

FINAL 

CLUSTER 

START 

FINISH 

GROUPS 

GROUP 

STAGE 

CLUSTER 

GROUP 

126 

127 

91.7 

91.7 

0443 

0002 

A 

128 

129 

90.7 

90.7 

0440 

0002 

A 

132 

133 

89.7 

89.7 

0437 

0002 

A 

134 

135 

92.8  ' 

92.8 

0445 

0002 

A 

138 

139 

86.7 

86.7 

0417 

0002 

B 

140 

141 

86.3 

86.3 

0414 

0002 

C 

143 

144 

82.8 

82.8 

0378 

0002 

C 

146 

147 

87.6 

87.6 

0422 

0002 

C 

151 

152 

86.3 

86.3 

0412 

0002 

D 

154 

155 

90.0 

90.0 

0439 

0002 

E 

157 

158 

89.9 

89.9 

0438 

0002 

E 

159 

160 

84,7 

84.7 

0397 

0002 

E 

163 

164 

85.2 

85.2 

0403 

0002 

F 

167 

168 

88.2 

88.2 

0429 

0002 

F 

171 

172 

87.9 

87.9 

0426 

0002 

F 

176 

177 

81.6 

81.6 

0367 

0002 

G 

182 

183 

88.2 

88.2 

0428 

0002 

H 

125 

127 

88.1 

89.4 

0427 

0003 

A 

128 

130 

88.4 

89.2 

0431 

0003 

A 

131 

133 

87.7 

88.3 

0424 

0003 

A 

134 

136 

89.6 

90.7 

0436 

0003 

A 

137 

139 

85.7 

86.1 

0405 

0003 

B 

140 

142 

85.0 

85.5 

0402 

0003 

C 

146 

148 

83.2 

84.7 

0382 

0003 

C 

150 

152 

84.1 

84.9 

0393 

0003 

D 

153 

155 

87.3 

88.3 

0421 

0003 

E 

156 

158 

89.2 

89.5 

0434 

0003 

E 

156 

158 

89.2 

89.5 

0434 

0003 

E 

159 

161 

82.6 

83.4 

0375 

0003 

E 

162 

164 

81.4 

82.8 

0365 

0003 

F 

167 

169 

84.4 

85.6 

0395 

0003 

F 

171 

175 

80.3 

82.9 

0355 

0003 

F 

176 

178 

78.6 

79.6 

0344 

0003 

G 

181 

183 

85.3 

86.2 

0404 

0003 

H 

149 

152 

82.6 

83.7 

0376 

0004 

D 

162 

165 

75.3 

78.9 

0321 

0004 

F 

167 

170 

80.4 

83.1 

0357 

0004 

F 

176 

179 

69.2 

74.4 

0288 

0004 

G 

180 

183 

75.9 

81.1 

0326 

0004 

H 

140 

144 

79.4 

81.5 

0350 

0005 

C 

162 

166 

73.0 

76.6 

0308 

0005 

F 

180 

184 

70-2 

76.8 

0298 

0005 

H 

125 

130 

86.1 

87.3 

0409 

0006 

A 

131 

136 

86-9 

88.0 

0418 

0006 

A 

140 

145 

78.2 

80.4 

0340 

0006 

C 

153 

158 

83.0 

85.4 

0380 

0006 

E 

167 

173 

76.9 

79.5 

0333 

0007 

F 

140 

148 

77.5 

79.3 

0336 

0009 

C 
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CO-PERF 


BETWEEN 

CO-PERF 

#  TASKS 

FINAL 

TASK  RANGE 

MERGING 

WITHBSr 

CLUSTERING 

IN 

CLUSTER 

START 

FINISH 

GROUPS 

GROUP 

STAGE 

CLUSTER 

GROUP 

153 

161 

76.7 

80.9 

0332 

0009 

E 

176 

184 

59.5 

66.8 

0245 

0009 

G-H 

125 

136 

86,0 

86.8 

0407 

0012 

A 

162 

173 

69.6 

73.8 

0292 

0012 

F 

124 

136 

83.2 

86.2 

0384 

0013 

A 

149 

161 

75.0 

78.3 

0317 

0013 

D-E 

162 

174 

64.4 

72.4 

0266 

0013 

F 

162 

175 

63.3 

71,0 

0263 

0014 

F 

124 

139 

82.3 

85.0 

0372 

0016 

A-B 

120 

139 

73.2 

81.0 

0310 

0020 

A-B,X* 

120 

148 

67.8 

74.9 

0282 

0029 

A-C,X 

120 

161 

66.1 

71.4 

0278 

0042 

A-E,X 

120 

175 

60.1 

67.0 

0250 

0056 

A-F,X 

116 

175 

59.4 

66.0 

0244 

0060 

A-F,X,Z* 

116 

184 

52.3 

63.0 

0213 

0069 

A-H,X,Z 

Co-performance  measures  are  not  shown  for  groups  X  and  Z  until  they  are  combined  with  other  groups. 


Table  D-3.  Averse  Co-Performance  from  TASSET 


CODAP 

T-PATH 

GROUP  NUMBER 

A  125 

126 

127 

128 

129 

130 

131 

132 

133 

134 

135 

136 
124 

B  137 

138 

139 

C  140 

141 

142 

143 

144 

145 

146 

147 

148 


OS  TASK 

TASSET 

Average 

NUMBER 

CO-PERF 

J407 

85.47 

J410 

86.75 

J412 

86.73 

J436 

87.22 

J439 

86.69 

J441 

85.13 

J420 

85.91 

J432 

86.31 

J449 

86,51 

J424 

86.59 

J426 

87.78 

J427 

86.39 

J405 

83.23 

J415 

85.72 

J428 

86.23 

J443 

86.15 

J411 

80.32 

J414 

80.60 

J413 

81.28 

J425 

78.55 

J433 

78.38 

J421 

76.90 

J437 

78.60 

J438 

78.78 

J440 

80.47 

149 

J406 

82.60 

150 

J408 

84.13 

151 

J434 

83.76 

152 

J442 

84.41 

153 

J409 

81.72 

154 

J431 

82.22 

155 

J448 

82.36 

156 

J419 

82.01 

157 

J430 

82.17 

158 

J447 

82.09 

159 

J417 

76.80 

160 

J446 

78.14 

161 

J429 

80.14 
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APPENDIX  E.  ESTIMATING  TASK  PERFORMANCE  TIME  FOR  THE 

SECONDARY  AFS 


E.l.  Introduction 


When  substituting  cross  trained  personnel  into  maintenance  crews  for  their  secondary  aircraft, 
one  must  consider  how  much  longer,  if  at  all,  those  personnel  take  to  perform  a  maintenance  task  than 
do  personnel  who  perform  the  task  as  a  routine  responsibility  of  their  primary  AFS. 

To  answer  this  question,  we  took  the  following  steps.  First,  we  searched  for  existing  sources  of 
primary  and  secondary  AFS  task  performance  times.  We  found  that  the  SUMMA  studies  had  gathered 
SME  estimates  of  primary  and  secondary  AFS  task  performance  times  for  the  F-18  and  the  F-16.^* 
Second,  we  looked  for  measures  of  other  factors  likely  to  affect  task  performance  times,  i.e.,  task  type 
(e.g.  minor  repair,  remove  and  replace,  troubleshoot,  or  verify),  task  learning  difficulty,  and  specialty 
knowledge  (e.g.  mechanical,  computer,  fluids  and  gases,  etc.).  We  found  these  data  in  the  F-18  and  F-16 
SUMMA  data  bases  and  in  the  F-16  OS  data  base.  Finally,  we  used  regression  analysis  to  identify 
relationships  between  secondary  AFS  task  performance  times  and  other  quantifiable  factors.  The 
regressions,  listed  below,  are  detailed  in  the  following  sections: 


F-18  Data  Analyses: 


F-16  Data  Analyses: 


•  Regression  of  secondary  time  versus  primary  time 

•  Regression  of  mean-offset  versus  primary  time 

•  Regression  of  secondary  time  versus  primary  time  and 
task  learning  difficulty 

•  Regression  of  secondary  time  versus  primary  time  and 
specialty  knowledge  factors 

•  Regression  of  secondary  time  versus  primary  time 

•  Regression  of  secondary  time  versus  primary  time  and 
task  learning  difficulty  (SUMMA) 

•  Regression  of  secondary  time  versus  primary  time  and 
weighted  task  learning  difficulty  (OS) 


We  planned  to  study  primary  and  secondary  task  performance  times  for  both  aircraft,  ATFs  and  F-15s,  in 
the  proof  of  concept  smdy.  However,  F-15  data  were  unavailable,  so  we  revised  our  study  plan  to  include  the  F-16, 
for  which  a  variety  of  high  quality  data  exist. 
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E.2.  F-18 


We  based  our  initial  attempts  to  estimate  secondary  AFS  task  performance  times  on  analyses  of 
collected  as  part  of  a  SUMMA  study  conducted  for  the  ATF  in  the  mid-1980s.  In  that  study,  the 
ATF’s  ICNIA  systems  were  represented  by  similar  systems  on  the  Navy’s  F-18.  F-18  Electrician’s  Mates 
at  Cecil  Field,  Florida  estimated  primary  and  secondary  AFS  task  performance  times  and  recorded  their 
responses  on  a  slightly  modified  version  of  the  data  collection  instrument  described  in  (Lamb,  1987). 
The  Electrician’s  Mates  answered  26  questions  about  127  maintenance  actions;  two  to  four  respondents 
considered  each  maintenance  action.  The  maintenance  actions  included  11  minor  repair  (M)  actions,  48 
remove  and  replace  (R)  actions,  16  troubleshoot  (T)  actions  and  52  verify  (V)  actions.  The  survey 
responses  were  aggregated  as  described  in  the  referenced  report.  The  resulting  data  base  included  mean 
respondent  estimates  of  task  performance  time  by  primary  AFS,  task  performance  time  by  secondary 
AFS,  task  learning  difficulty,  and  required  mechanical,  computer,  electronics,  fluids  and  gases,  and 
metals  characteristics  knowledge. 

On  review  of  the  data,  we  found  that  43  of  the  127  tasks  had  shorter  secondary  performance 
times  than  primary  performance  times.  We  have  assumed  that  secondary  AFS  personnel  will  perform 
each  task  less  frequently  than  primary  AFS  personnel  and  so  will  take  longer  to  perform  it.  While  there 
are  several  explanations  for  data  showing  short  secondary  performance  times,  e.g.  respondent  error, 
inaccurately  stated  survey  instructions,  or  failure  to  properly  account  for  secondary  AFS  inability  to 
perform  the  task,  all  suggest  survey  problems  rather  than  incompatibility  with  our  assumption.  Since 
we  have  no  explanation  for  short  secondary  task  performance  times  other  than  the  possibility  of  data 
collection  problems,  we  developed  two  modifications  of  secondary  performance  time,  seca_hrs  and 
secb_hrs,  for  use  in  our  analyses. 

Original  data:  sec_hrs  =  average  estimated  performance  time  for  secondary  AFS 

Modification!:  seca_hrs  =  max{sec_hrs,prim_hrs}  (43  of  127  modified) 

Modification  2:  If  sec_hrs<  .Sprim  hrs  (9  of  127  modified) 

secb_hrs  =  prim_hrs 


Else 

secb_hrs  =  sec_hrs 

Regression  of  secondary  time  versus  primary  time:  Our  first  analysis  considered  the 
relationship  of  secondary  AFS  task  performance  time  to  primary  AFS  task  performance  time  (prim_hrs). 
Regressions  for  the  three  versions  of  secondary  performance  time,  with  and  without  intercepts,  are 
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Table  E^l.  F-18  R^ession  Results 


Secondary  AFS 
Performance  Time  or 
Surrogate 

Intercept 

Coefficient  of 
Primary  AFS 
Task 

Performance 

Time 

(SUMMA) 

Coefficient  of 
Task  Learning 
Difficulty 
(SUMMA) 

r^ 

1. 

sec_hrs 

.7791 

0.3897 

.3538 

2. 

sec_hrs 

0.5019 

.5233 

3. 

seca_hrs 

.1561 

1.0210 

.9444 

4. 

secahrs 

1.0435 

.9602 

5. 

secb_hrs 

.1952 

0.9512 

.9114 

6. 

secb_hrs 

0.9793 

.9367 

7. 

secb_hrs  (M  tasks) 

0.9620 

.9381 

8. 

secb_hrs  (R  tasks) 

1.0576 

.9449 

9. 

secb_hrs  (T  tasks) 

1.1943 

.8768 

10. 

secb_hrs  (V  tasks) 

0.8712 

.9557 

11. 

m-offset 

.1877 

-0.0430 

.0133 

12. 

m-offset  (M  tasks) 

.5719 

-0.1752 

.0203 

13. 

m-offset  (R  tasks) 

.2420 

-0.0431 

.0833 

14. 

m-offset  (T  tasks) 

.5126 

-0.1078 

.0044 

15. 

m-offset  (V  tasks) 

.0230 

-0.0419 

.0073 

16. 

secb_hrs 

-1.7462 

1.3576 

.0781 

17. 

secb_hrs 

.1578 

0.9504 

0.0139 

.9114 

displayed  in  Table  E-1,  equations  1  through  6.  Notice  that  the  equations  with  intercepts,  in  addition  to 
having  slightly  lower  r^  values,  make  less  sense.  That  is,  when  the  primary  AFS  task  performance  time 
is  zero,  we  expect  the  secondary  performance  time  to  be  zero  as  well,  giving  an  equation  with  no 
intercept.  Also  notice  the  regressions  performed  with  the  modified  data  seca  hrs  and  secb_hrs.  The 
for  the  seca_hrs  regression  is  slightly  higher  than  that  for  the  secb_hrs  regression,  as  expected  based  on 
the  modification  to  the  data.  However,  we  prefer  to  use  the  secb  hrs  data.  The  modification  to  create 
that  data  set  affected  only  9  points  where  secondary  performance  time  was  less  than  half  primary 
performance  time  and  in  our  opinion,  where  the  survey  error  explanation  was  more  suitable. 

We  also  evaluated  secondary  AFS  task  performance  time  versus  primary  AFS  task  performance 
time  by  maintenance  action  category.  We  regressed  secondary  performance  time  against  primary 
performance  time  in  the  categories  M  (minor  repair),  R  (remove/replace),  T  (troubleshoot),  and  V 
(Verify).  While  the  data  indicate  that  use  of  maintenance  action  category  increases  predictive  ability  in 
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some  categories,  they  do  not  do  so  for  all  categories.  We  elected  not  to  pursue  this  path  at  this  time. 
(See  Table  E-1,  equations  7  through  10.) 

Regression  of  mean-offset  versus  primary  time:  We  also  considered  the  predictive  ability  of 
the  mean-offset  (m-offset),  defined  as  the  secondary  AFS  mean  task  performance  time  less  the  primary 
APS  mean  task  performance  time,  expressed  in  standard  deviations  from  the  primary  AFS  time: 

m-offset  =  (sec_hrs  -  prim_hrs)/(stdpri,„ jJ 

We  also  performed  these  regressions  by  task  category,  obtaining  the  results  in  Table  E.6,  equations  1 1 
through  15.  We  did  not  consider  the  predictive  ability  of  the  mean-offset  worth  further  analysis. 

Regression  of  secondary  time  versus  primary  time  and  task  difficulty:  We  evaluated  the 
predictive  ability  of  task  difficulty  using  the  mean  of  normalized  SME  task  difficulty  estimates  (diff)  from 
the  F-18  SUMMA  survey.  We  also  considered  the  combined  predictive  ability  of  primary  AFS  task 
performance  time  (prim_hrs)  and  task  difficulty  (diff).  (See  Table  E.l,  equations  16  and  17  for  the 
results.)  Adding  the  difficulty  measure  did  not  increase  the  r^.  In  fact,  the  r^  adjusted  for  degrees  of 
freedom  is  lower  than  the  r^  for  the  model  using  prim  hrs  alone.  These  results  discourage  use  of 
SUMMA  task  difficulty  estimates  as  a  task  performance  time  predictor. 

Regression  of  secondary  time  versus  primary  time  and  specialty  knowledge  factors: 

During  the  SUMMA 
survey,  subject  matter 
experts  estimated  the 
specialty  knowledge 
required  to  perform  each 
task.  The  experts 
estimated  specialty 
knowledge  requirements 
in  the  areas  of 
computers,  electronics, 
fluids  and  gases,  metals, 

and  mechanics.  Their  estimates  were  made  on  a  7  point  scale;  they  were  subsequently  normalized  to  give 
a  mean  requirement  of  4.  Convinced  of  the  value  of  primary  AFS  task  performance  time  in  predicting 
secondary  AFS  task  performance  time,  we  looked  at  the  incremental  increase  to  the  regression’s 
predictive  ability  brought  by  each  specialty  knowledge  parameter.  Table  E-2  shows  the  incremental 


Table  E-2.  Specialty  Knowledge  Variables 


Variable  Added 

Partial  r^ 

Model 

prim_hrs 

0.9112 

0.9112 

Mechanics 

0.0012 

0.9124 

Metals 

0.0004 

0.9128 

Fluids 

0.0001 

0.9129 

Electronics 

0.0000 

0.9129 

Computer 

0.0000 

0.9130 
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increase  of  when  each  of  the  specialty  knowledge  variables  are  added  to  the  modeF: 

secb_hrs  =  intercept  +  prim_hrs 

In  all  cases,  the  partial  r^  from  the  added  variable  was  less  than  .0012.  We  did  not  pursue  use  of 
specialty  knowledge  for  predicting  task  performance  time. 

Discussion:  Several  F-18  data  issues  were  of  concern.  First,  we  questioned  the  quality  of  the 
F-18  survey  data  and  were  unable  to  verify  it.  The  data  had  several  weaknesses; 

•  The  F-18  SUMMA  results  are  based  on  data  from  a  small  number  of  respondents. 

•  The  data  include  many  outliers  but  the  survey  had  no  cross  checks  or  other  information  for 

eliminating  or  "scrubbing"  them. 

•  A  third  of  the  original  task  data  sets  showed  secondary  AFS  task  performance  time  less  than 

primary  AFS  task  performance  time. 

Empirical  data  for  task  performance  times  were  not  collected  for  comparison  with  the  survey,  and 
historical  data  are  not  readily  available  for  comparison  with  the  survey  task  data.  Thus  task  performance 
time  data  are  imverified. 

A  second  concern  is  the  possible  bias  in  the  survey  data  where  respondents  may  supply  the 
answers  they  think  the  researchers  want.  In  this  situation.  Electrician’s  Mates  might  think  someone  else 
should  perform  the  tasks,  so  they  estimate  shorter  task  performance  times  in  order  to  influence  the 
researchers  toward  giving  the  tasks  to  another  rating  (AFS).  Finally,  both  survey  instructions  and 
intended  responses  can  be  misunderstood.  For  example,  failure  to  supply  a  secondary  AFS  performance 
time  can  be  an  omission  or  an  indicator  that  a  secondary  AFS  cannot  perform  the  task. 

Based  on  F-18  SUMMA  survey  data,  the  best  predictor  of  secondary  AFS  performance  time  is 
primary  AFS  performance  time.  However,  we  were  concerned  about  the  accuracy  and  reliability  of  our 
results  1)  because  of  the  survey’s  small  sample  size  and  wide  ranging  numbers  and  2)  because  the  slope 
suggested  that  personnel  in  a  secondary  AFS  performed  a  task  faster  than  personnel  in  the  primary  AFS. 
Suspecting  that  the  slope  might  in  fact  be  one,  we  performed  a  t-test  on  the  data.  Our  results  indicated 
that  at  a  95%  confidence  interval,  the  slope  includes  1.0,  so  our  choice  for  representing  secondary  AFS 
performance  time  for  the  F-18  is  the  equation  below: 

secondary  hours  =  primary  hours 


The  regression  for  secb_hrs  as  a  function  of  prim_hrs  has  a  slightly  different  r^  here  because  one  point  with 
missing  values  for  the  specialty  knowledge  variables  was  dropped  from  the  regression. 
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E.3.  F-16 


We  expanded  our  study  to  include  analyses  of  F-16  data  because  1)  the  F-18  SUMMA  data  were 
suspect,  as  discussed  above,  2)  F-15  data  for  the  analyses  were  unavailable,  and  3)  F-16  data  were  readily 
available  and  wide  ranging.  We  used  primary  and  secondary  AFS  task  performance  time  data  from  the 
F-16  SUMMA  survey,  reported  in  (Lamb,  1987).  The  F-16  study  covered  only  36  LCOM  COMM/NAV 
tasks;  however,  approximately  40  SMEs  responded  to  the  survey,  with  20-24  addressing  each  task.  We 
converted  SUMMA  survey  data  to  mean  response  for  primary  AFS  performance  time,  secondary  AFS 
performance  time,  and  task  difficulty.  We  attempted  to  validate  the  SME  estimated  performance  times 
using  LCOM  data  but  our  data  set  did  not  have  historical  performance  times  for  the  COMM/NAV  tasks 
in  the  SUMMA  survey. 


Table  El-3.  F-16  Regression  Results 


Secondary 

AFS 

Performance 

Time 

Intercept 

Coefficient  of 
Primary  AFS 
Task 

Performance 
Time  (SUMMA) 

Coefficient  of 
Task 
Learning 
Difficulty 
(SUMMA) 

Coefficient  of 
Weighted  Task 
Learning 
Difficulty  (OS) 

r^ 

sec_hrs 

-0.0776 

1.33 

.9553 

sec_hrs 

1.24 

.9882 

sec_hrs 

-1.4023 

0.8133 

.5609 

4. 

sechrs 

0.3166 

.8463 

5. 

sec_hrs 

0.1061 

1.42 

-0.0888' 

.9578 

6. 

sec_hrs 

1.38 

-0.0413 

.9899 

7. 

sec  hrs 

-0.5484 

.3413 

.3030 

8. 

sec_hrs 

.2264 

.8493 

9. 

sec_hrs 

-0.04323 

1.38 

-0.0092 

.9791 

10. 

sec_hrs 

1.38 

-0.0191 

.9957 

Regression  of  secondary  time  versus  primary  time:  Our  first  analysis  considered  the 
relationship  of  the  task  performance  time  by  the  secondary  AFS  (sec_hrs)  to  that  by  the  primary  AFS 
(prim_hrs).  Reviewing  the  data,  we  found  only  1  of  the  36  data  points  with  a  secondary  performance 
time  less  than  the  primary  performance  time.  Therefore,  we  made  no  modification  to  secondary 
performance  times. 

Regressions  of  secondary  performance  time  against  primary  performance  time,  both  with  and 
without  an  intercept,  are  displayed  in  Table  E-3.  As  with  the  F-18  analyses,  we  prefer  the  no-intercept 
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regression  because  it  makes  intuitive  sense.  (See  Table  E-3,  equations  1  and  2.) 


Regression  of  secondary  time  versus  primary  time  and  task  difficulty  (SUMMA):  In  the  first 
portion  of  this  analysis  we  considered  the  predictive  ability  of  task  difficulty,  using  the  mean  of 
normalized  SME  estimates  of  task  difficulty  (diff)  generated  during  the  F-16  SUMMA  survey.  The 
results,  in  equations  3  and  4  of  Table  E-3,  have  lower  values  than  the  regressions  against  primary  time 
and  so  discourage  use  of  SUMMA  estimates  of  task  difficulty  alone  as  a  task  performance  time  predictor. 

We  also  considered  the  combined  predictive  ability  of  primary  APS  task  performance  time  and 
task  difficulty  (diff).  (See  equations  5  and  6  of  Table  E.8.)  Addition  of  the  difficulty  measure  increases 
the  r^  slightly. 

Regression  of  secondary  time  versus  primary  time  and  weighted  task  learning  difficulty  (OS): 
Another  way  to  add  task  difficulty  to  the  model  is  to  use  task  learning  difficulty  (TLD)  data  from  the  OS 
data  bases.  If,  in  fact,  TLD  is  a  good  predictor  of  secondary  APS  task  performance  time,  we  will  need 
TLD  data  for  all  systems  of  interest.  OS  data  for  TLD  are  available  for  all  Air  Force  systems  and  are 
reliably  based  on  hundreds  of  survey  responses,  where  SUMMA  survey  data  are  available  only  for 
selected  systems. 

Thus,  we  used  F-16  SUMMA  survey  data  in  conjunction  with  F-16  OS  data  as  another  way  to 
study  the  combined  predictive  ability  of  primary  AFS  task  performance  time  and  task  difficulty.  TLD 
data  are  available-by  OS  task  for  the  F-16  Avionics  AFS  452X1C  and  its  predecessor  326X8C.  We 
planned  to  use  OS  TLD  in  combination  with  SUMMA  survey  data,  identified  by  LOOM  task,  to  develop 
a  regression  as  in  the  section  above.  We  opted  to  use  the  older  AFS  326X8  because  we  had  a  map  of 
OS  tasks  to  LOOM  tasks  provided  developed  using  the  Semantic- Assisted  Analysis  Technology  (SAAT) 
(Metrica,  1991).  (See  Tables  E-4  and  E-5.)  However,  OS  tasks  do  not  match  the  maintenance  actions 
of  LOOM  tasks  on  a  one-to-one  basis,  as  discussed  in  Appendix  C.  We  applied  the  following  rules  in 
order  to  map  the  data  for  this  analysis: 

•  When  OS  tasks  were  mapped  to  several  levels  of  aggregation  or  WUCs,  the  mapping  to  the 
highest  level  LOOM  task  was  retained.  The  mappings  to  LOOM  tasks  at  a  more  detailed  level 
were  dropped. 

•  OS  tasks  which  did  not  have  a  MAAVUC  combination  matching  that  generated  by  an  LOOM 
task  were  ignored  in  this  study. 

These  matches  quickly  led  to  a  complicated  mapping  where  one  LOOM  task  could  map  to  one, 
many,  or  no  OS  tasks,  and  one  OS  task  could  map  to  one,  may,  or  no  LOOM  tasks.  (See  Tables  E-6 
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and  E-7.) 


With  the  mapping  complete,  it  was  necessary  to  allocate  the  OS  task  learning  difficulties  to  those 
LCOM  tasks  that  had  more  than  one  OS  task  mapped  to  them.  The  approach  we  took  was  to  weight  the 
task  learning  difficulty  of  each  OS  task  by  the  percent  time  spent  performing  the  task.^^  The  resulting 
weighted  task  learning  difficulty  (wgtdjd)  was  the  variable  we  used  in  the  regression.  The  regression 
results  in  Table  E-3,  equations  5  through  8,  are  similar  to  the  results  using  the  SUMMA  difficulty 
measures.  TLD  is  not  a  strong  predictor  on  its  own  and  does  not  contribute  enough  to  the  predictive 
ability  of  primary  time  to  make  it  a  worthwhile  addition  to  the  predictive  equation.  In  the  task  difficulty 
analyses,  the  two  SUMMA  and  OS  task  difficulty  measures  played  similar  roles.  However,  as  with  the 
F-18  analyses,  we  do  not  consider  task  learning  difficulty  to  be  a  worthwhile  addition  to  the  regression. 


E.4.  Summary 

In  general,  we  had  more  confidence  in  our  F-16  analyses  than  the  other  work.  The  F-16 
SUMMA  data  were  based  on  the  responses  of  20-24  SMEs  and  showed  more  consistency. 

Based  on  these  studies,  the  best  predictor  of  secondary  AFS  performance  time  is  primary  AFS 
performance  time.  At  a  95%  confidence  interval,  the  slope  does  not  include  1.0,  so  our  choice  for 
representing  secondary  AFS  performance  time  for  the  F-16  is  equation  2  in  Table  E-3,  which  is  repeated 
below: 


sec_hrs  =  1.24  prim_hrs  r^  =  .9882 


The  weighting  is  similar  to  that  used  in  computing  Average  Task  Difficulty  Per  Unit  Time  Spent 
(ATDPUTS). 
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Table  E-4.  Derivation  of  F-16  Weighted  Task  Learning  Difficulty  -  Step  1. 

F-16  OS  (Old)  Tasks  Identified  by  Metrica  SAAT  as  Involving  Work  Units  of  the  65  or  76  Systems 


OS 

TASK 

MAI NT  ACTION/ 

WUC 

TASK 

DIFFICULTY 

197 

V76E00 

5.4556 

199 

V76E00 

5.4126 

207 

_76EEG 

5.5549 

*224 

V76EB0 

1.8605 

224 

V76EA0 

1.8605 

644 

V65AAF 

2.4875 

651 

T65A00 

5-7402 

688 

T65A00 

5.5322 

689 

T65A00 

5.4297 

690 

T65A00 

5.3588 

691 

T65A00 

5.4302 

692 

T65A00 

5.2841 

693 

T65A00 

5.5276 

694 

T65A00 

5.0954 

695 

T65A00 

5.0584 

696 

T65A00 

4.9127 

697 

T65A00 

4.9254 

*713 

V65A00 

4.7198 

713 

V65AC0 

4-7198 

716 

V65A00 

4.8646 

723 

V65A00 

3.9026 

725 

V65A00 

4-7546 

726 

V65A00 

4.5106 

751 

T76C00 

5.4867 

764 

T76E00 

5.7311 

*765 

T76EC0 

6.0961 

765 

T76E00 

6.0961 

766 

T76E00 

6.0874 

767 

T76E00 

6.4003 

*768 

T76EC0 

5.7884 

768 

T76E00 

5.7884 

769 

T76E00 

5.8724 

771 

T76E00 

5.5319 

772 

T76E00 

6.7100 

778 

V76E00 

4.9496 

779 

V76ED0 

3.9828 

780 

V76E00 

4.3643 

785 

V76E00 

4.9671 

*789 

V76ED0 

4.3065 

789 

V76EH0 

4.3065 

789 

V76EE0 

4.3065 

*  INDICATES  ONE  OS  TASK  MAPPED  TO  MULTIPLE  MA/WUC 
COMBINATIONS. 
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Table  E-5.  Derivation  of  F-16  Weighted  Task  Learning  Difficulty  -  Step  2 
Reorder  F-16  OS  (Old)  Tasks  by  MA/WUC  Combination 


MAINT.  ACTION/ 

OS 

TASK 

wyc 

TASK 

DIFFICULTY 

_76EEG 

207 

5.5549 

T65AA0 

651 

5.7402 

T65A00 

688 

5.5322 

689 

5.4297 

690 

5.3588 

691 

5.4302 

692 

5.2841 

693 

5.5276 

694 

5.0954 

695 

5.0584 

696 

4.9127 

697 

4.9254 

T76C00 

751 

5.4867 

T76E00 

764 

5.7311 

765 

6.0961 

766 

6.0874 

767 

6.4003 

768 

5.7884 

769 

5.8724 

771 

5.5319 

772 

6.7100 

T76EC0 

765 

6.0961 

768 

5.7884 

V65A00 

713 

4.7198 

716 

4.8646 

723 

3.9026 

725 

4.7546 

726 

4-5106 

V65AAF 

644 

2.4875 

V65AC0 

713 

4.7198 

V76E00 

197 

5.4556 

199 

5.4126 

778 

4.9496 

780 

4.3643 

785 

4.9671 

V76EA0 

224 

1.8605 

V76EB0 

224 

1 .8605 

V76ED0 

779 

3.9828 

789 

4.3065 

V76EE0 

789 

4.3065 

V76EH0 

\ 

789 

4.3065 
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Table  E-6.  Derivation  of  F-16  Weighted  Task  Learning  Difficulty  -  Step  3. 
Consolidate  OS  Originating  MAAVUC  Combinations  to  LCOM  MAAVUC 
Combinations 


LCOM 

OS 

MAIN!  ACTION/ 

MAI NT  ACTION/ 

OS 

TASK 

WUC 

WUC 

.  imk 

DIFFICULTY 

T65A00 

T65AA0 

651 

5,7402 

T65A00 

688 

5,5322 

689 

5.4297 

690 

5,3588 

691 

5,4302 

692 

5.2841 

693 

5,5276 

694 

5,0954 

695 

5,0584 

696 

4,9127 

697 

4,9254 

V65A00 

V65A00 

713 

4,7198 

716 

4.8646 

723 

3.9026 

725 

4.7546 

726 

4.5106 

V65AAF 

644 

2.4875 

V65AC0 

713 

4,7198 

T76C00 

T76CO0 

751 

5,4867 

T76E00 

T76EO0 

764 

5,7311 

765 

6.0961 

766 

6.0874 

767 

6.4003 

768 

5.7884 

769 

5.8724 

771 

5.5319 

772 

6.7100 

T76EC0 

765 

6,0961 

768 

5,7884 

V76E00 

V76EO0 

197 

5,4556 

199 

5,4126 

778 

4.9496 

780 

4,3643 

785 

4,9671 

V76EA0 

224 

1,8605 

V76EB0 

224 

1.8605 

V76ED0 

779 

3,9828 

789 

4,3065 

V76EE0 

789 

4.3065 

V76EH0 

789 

4.3065 
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Table  E-7.  Derivation  of  F-16  Weighted  Task  Learning  Difficulty  -  Step  4. 
Eliminate  Duplicate  OS  Task  Mappings  to  Single  LOOM  Task;  and  Weight  TLD 
by  %  Time  Performing 


LCOM 

WUC/ 

OS 

TASK 

COMPOSITE 

TASK 

PRIM  TIME 

SEC  TIME 

ACTION 

TASK 

DIFF 

%TIME 

LRNG  DIFF 

* 

T65A00 

1,12273 

1.54091 

T65A00 

651 

5.7402 

0.2114 

5.1891 

T65A00 

688 

5.5322 

0.3622 

T65AOO 

689 

5,4297 

0.1738 

T65A00 

690 

5.3588 

0,1975 

T65A00 

691 

5.4302 

0,2129 

T65A00 

692 

5,2841 

0.398 

T65A00 

693 

5,5276 

0.1271 

T65A00 

694 

5.0954 

0.4589 

T65A00 

695 

5.0584 

0,441 

T65A00 

696 

4.9127 

0.5584 

T65A00 

697 

4.9254 

0,7334 

R65AA0 

0,42273 

0,47727 

* 

V65A00 

0,62273 

0.80455 

V65AAF 

644 

2.4875 

1.2401 

3,5953 

V65AC0 

713 

4.7198 

0.1409 

V65A00 

716 

4,8646 

0.2303 

V65A00 

723 

3,9026 

0.34182 

V65A00 

725 

4.7546 

0.1944 

V65A00 

726 

4.5106 

0,6474 

* 

T76C00 

0,98889 

1,17222 

T76C00 

751 

5,4867 

0,3571 

5,4867 

R76CD0 

0,57778 

0.71667 

R76CE0 

0,85833 

0,95 

V76C00 

0.295 

0.38 

* 

T76E00 

1,53333 

1,9625 

T76E00‘ 

764 

5,7311 

0,7719 

5.9624 

T76EC0 

765 

6.0961 

0,4966 

T76EO0 

766 

6,0874 

0,5586 

T76E00 

767 

6,4003 

0,204 

T76EC0 

768 

5.7884 

0.5241 

T76E00 

769 

5,8724 

0.6176 

T76E00 

771 

5.5319 

0.5768 

T76E00 

772 

6,71 

0.4654 

R76ED0 

0,85417 

0,625 

R76EE0 

0,55 

0,6125 

R76EG0 

0,44167 

0,50417 

R76EK0 

0,56667 

0.7 

* 

V76E00 

0.80417 

1,02083 

V76E00 

199 

5,4126 

0.2125 

4,5843 

V76E00 

197 

5.4556 

0.9886 

V76EA0 

224 

1.8605 

0,3932 

V76E00 

778 

4.9496 

0-321 

V76ED0 

779 

3.9828 

0,4404 

V76E00 

780 

4,3643 

0.6166 

V76E00 

785 

4.9671 

0,8878 

V76ED0 

789 

4.3065 

0.0833 

*  DATA  POINTS  USED  IN  REGRESSIONS  OF  SECONDARY  TIME  AGAINST  WEIGHTED  TASK  LEARNING 
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APPENDIX  F.  ESTIMATING  TRAINING  REQUIREMENTS 


F.I.  Introduction 

F-15  SMEs  were  asked  to  evaluate  the  training  requirements  for  5-level  ATF  avionics  maintainers 
being  cross  trained  to  maintain  F-15  UHF  and  IFF  equipment.  The  SMEs  were  given  detailed  outlines 
of  the  contents  of  the  three  F-15  courses,  the  Initial  Skills  Course  (Lowry  AFB),  the  FTD  course  (Eglin 
AFB),  and  OJT.  They  were  asked  to  identify  any  training  the  ATF  maintainers  would  require  and  the 
total  hours  or  a  percentage  of  the  F-15  training  hours  that  would  be  needed  for  the  cross  training.  The 
results  of  these  surveys  are  shown  in  Sections  F.2.  (Initial  Skills  Course),  F.3.  (FTD  Course),  and 
F.4.(OJT) 

Additionally ,  a  top-down  approach  was  used  to  confirm  the  OJT  requirement.  Section  F.5  shows 
the  questionnaires  which  asked  the  SMEs  for  the  percentages  of  their  total  time  in  OJT  allocated  to  UHF 
and  IFF  systems  and  subsystems.  The  results  of  this  approach  are  also  displayed  in  Section  F.5. 

Note  that  the  data  collected  through  these  surveys  are  sparse  and  so  cannot  be  used  to  reach  firm 
conclusions  regarding  cross  training  requirements.  However,  the  process  followed  to  collect  these  data 
was  a  viable  one  which  if  followed  with  a  larger  SME  sample,  would  generate  more  precise  data. 
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F.2.  Program  of  Instruction,  Initial  Skills  Course  for  AFS  452X1C 

POI G3AQR45231C-000,  F-15  AVIONIC  COMMUNICATION,  NAVIGATION,  AND  PENETRATION 
AIDS  SYSTEMS  SPECIALTY  (Dated  30  June  1989,  LOWRY  TECHNICAL  TRAINING  CENTER),  is  presented 
below  with  the  following  additions.  POI  objectives  are  displayed  with  projected  course  hours  (classroom  hours/self¬ 
study  hours)  in  the  column  POI  HRS.  Hours  for  maintainers  from  another  aircraft  cross  training  to  maintain  F-15 
UHF,  IFF,  and  related  systems,  as  estimated  by  four  SMEs,  are  displayed  in  columns  #1  HRS,  #2  HRS,  #3  HRS, 
and  #4  HRS.  The  mean  and  standard  deviation  of  the  survey  responses  are  displayed  in  the  last  two  columns. 

Hours  associated  with  an  individual  line  item  of  the  POI  are  listed  in  parentheses.  These  hours  are  totaled 
and  displayed  adjacent  to  POI  subheadings.  The  hours  associated  with  the  subheadings  are  totaled  and  displayed 
in  bold  print  adjacent  to  the  POI  major  headings. 


SUMMARY:  SME  ESTIMATES  FOR  INITIAL  SKILLS  COURSE,  AFS  452X1C 


COURSE  ITEM 

POI 

HRS 

#1 

HRS 

#2 

HRS 

#3 

HRS 

#4 

HRS 

MEAN 

HRS 

STD 

DEV 

I.  ELECTRONIC  PRINCIPLES 

n.  INTRODUCTION  TO 
MAINTENANCE 

24.5/8 

0/0 

0/0 

0/0 

0/0 

ra.  GENERAL  MAINTENANCE 
PROCEDURES 

34.5/ 

12 

0/0 

4.275/ 

1.95 

4.6/4.5 

6.925/ 

2.0 

3.95/ 

2.11 

2.50/ 

1.60 

IV.  COMM/NAVAND 
PENETRATION  AIDS  SYSTEM  I 

58.5/ 

20 

11.125/ 

4.5 

47.25/ 

17 

34/ 

12.75 

26.25/ 

7.5 

29.75/ 

10.44 

13.20/ 

4.80 

V.  COMM/NAVAND 
PENETRATION  AIDS  SYSTEM  n 

34.5/ 

12 

6.75/ 

2.75 

17.25/ 

6.75 

15.0/ 

5.75 

20.5/ 

6.5 

14.88/ 

5.44 

5.08/ 

1.59 

VI.  TACTICAL  ELECTRONIC 
WARFARE  SYSTEM 

22.5/8 

0/0 

0 

0 

0 

Vn.  PENETRATION  AIDS 
SYSTEM 

26/8 

0/0 

0 

0 

4/1 

1.0 

1.73 

TOTAL  HOURS 

200.5/ 

68 

17.88/ 

7.25 

69.15/ 

25.7 

53.6/ 

23.0 

57.7/ 

17 

49.6/ 

18.2 

19.2/ 

7.1 
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DETAIL:  SME  ESTIMATES  FOR  ZNITIAL  SKILLS  COURSE,  AES  452X1C 


COURSE  ITEM 

POI 

HRS 

#1 

HRS 

n 

HRS 

#3 

HRS 

#4 

HRS 

MEAN 

HRS 

STD 

DEV 

I.  ELECTRONIC  PRINCIPLES 

n.  INTRODUCTION  TO 
MAINTENANCE 

24.5/8 

0/0 

0/0 

0/0 

0/0 

n.l.  Orientation 

1/0 

0/0 

0/0 

0/0 

0/0 

n.2.  Career  Ladder  Structure 

2.5/0 

0/0 

0/0 

0/0 

0/0 

IL2.a.  Given  an  Air  Force  Specialty 
Code  with  a  suffix,  identify  the 
elements  of  the  AFSC. 

(.5/0) 

II.2.b.  Given  a  list  of  statements, 
identify  the  duties  of  the  3,  5,  and  7 
skill  levels. 

(2/0) 

n.3.  Specific  Vulnerabilities  of 

AFSC 

1.5/0 

0/0 

0/0 

0/0 

0/0 

IL3.a.  Given  scenarios  with 

Operations  Security  vulnerabilities, 
identify  the  specific  vulnerabilities  of 
the  452xx  career  field  described  in 
each  scenario. 

•  - 

n.4.  Physical  Security 

2.5/1 

0/0 

0/0 

0/0 

0/0 

II. 4. a.  Given  a  list  of  statements, 
select  the  statements  that  pertain  to 
physical  security. 

n.5.  AF  Occupational  Safety  and 
Health  (AFOSH) 

8/3 

0/0 

0/0 

0/0 

0/0 

II.5.a.  Given  a  list  of  AF 

Occupational  Safety  and  Health 
(AFOSH)  Program  standards,  identify 
the  AFOSH  standards  for  the  452xx 
career  field. 

(.5/1) 

Il.S.b.  Given  a  list  of  hazards, 
identify  the  hazards  of  Radio 

Frequency  (RF)  energy. 

(.5/0) 

IL5.C.  Given  extracts  from  AFOSH 
standard  161-9,  identify  the  steps  in 
reporting  suspected  RF  overexposure. 

(1/0) 

II.5.d.  Given  a  list  of  safety 
practices,  identify  the  safety  practices 
used  with  RF  sources. 

(.5/0) 
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COURSE  ITEM 

POI 

#1 

#2 

#3 

#4 

MEAN 

STD 

HRS 

HRS 

HRS 

HRS 

HRS 

HRS 

DEV 

Il.S.e.  Given  a  list  of  safety 
practices,  identify  the  safety  practices 
used  with  compressed  gases. 


Il.S.f.  Given  a  list  of  safety  practices, 
identify  the  safety  practices  used  with 
electrical  power. 


Il.S.g.  Given  a  list  of  safety 
practices,  identify  the  safety  practices 
used  with  hydraulic  power. 


Il.S.h.  Given  a  list  of  safety 
practices,  identify  the  safety  practices 
used  with  hazardous  liquids 


II.5.i.  Given  a  list  of  safety  practices, 
identify  the  safety  practices  used  with 
portable  fire  extinguishers. 


Il.S.j.  Given  a  list  of  safety  practices, 
identify  the  safety  practices  used  with 
high  intensity  sound. 


Il.S.k.  Given  a  list  of  statements, 
identify  the  statements  that  describe 
the  procedures  in  keeping  the  work 
area  clean  and  safe. 


11.5.1.  Given  a  list  of  statements, 
identify  the  statements  that  describe 
the  procedures  in  preventing  Foreign 
Object  Damage  (FOD). 


11.5. m.  Given  a  list  of  statements, 
identify  the  statements  that  describe 
hydrazine  hazards. 


n.6.  Supply  Discipline 


II.6.a.  Given  a  list  of  statements, 
identify  the  statements  as  pertaining  to 
either  property  accountability  or 
property  responsibility. 


II.6.b.  Given  examples  yellow,  , 
green,  and  red  conditions  tags,  and 
scenarios  involving  equipment  status, 
complete  the  proper  tag  for  each 
scenario. 


n.7.  Maintenance  Management 


(.5/.5) 


(.57.5) 


0/0  0/0 


(1.5/0) 


0/0  0/0 


COURSE  ITEM 

POI 

HRS 

#1 

HRS 

#2 

HRS 

#3 

HRS 

#4 

HRS 

MEAN 

HRS 

STD 

DEV 

II.T.a.  Given  a  list  of  functions  and 
responsibilities,  identify  the  functions 
and  responsibilities  of  the  Deputy 
Commander  (DCM)  for  maintenance 
and  the  DCM  staff  agencies. 

(2.25/ 

1) 

n.7.b.  Given  a  list  of  statements, 
identify  the  statements  that  describe 
Maintenance  Data  Collection  (MDC). 

(2.25/ 

1) 

II.7.C.  Given  a  list  of  statements, 
identify  the  statements  that  pertain  to 
processing  and  controlling  material. 

(2/2) 

m.  GENERAL  MAINTENANCE 
PROCEDURES 

34.5/ 

12 

0/0 

4.275/ 

1.95 

4.6/4.5 

6.925/ 

2.0 

3.95/ 

2.11 

2.50/ 

1.60 

m.l.  Technical  Publications 

10.5/2 

0/0 

1.125/ 

.5 

3.6/4 

2.625/ 

1.0 

1.84/ 

1.38 

1.38/ 

1.56 

III.  1  .a.  Given  a  list  of  statements, 
select  the  statements  that  describe  the 
function  or  application  of  technical 
publications. 

(3/0) 

III.  1  .b.  Given  applicable  TOs  and 
statements  which  require  locating 
technical  information  to  complete  a 
task,  identify  where  the  information  is 
located. 

(4.5/2) 

(1.125/ 

.5) 

(3.6/4) 

III.  1  .c.  Given  two  technical  order 
(TO)  numbers  and  TO  indexes  0-1- 
01,0-1-02,  and  0-1-4;  locate  the  title, 
basic  date,  latest  change  date,  and 
supplement  for  the  TOs. 

(1.5/0) 

Ill.l.d.  Given  a  list  of  statements, 
identify  the  statements  that  describe  . 
Computer  Program  Identification 
Numbers  (CPIN). 

(1.5/0) 

III.2.  Maintenance,  Inspection 
Systems  and  Forms 

12/4 

0/0 

1.05/.2 

0/0 

0/0 

.26/.05 

.45/.0 

9 

III.2.a.  Given  a  list  of  statements; 
identify  the  level  of  maintenance 
described  in  each  statement. 

(1/0) 

(.25/0) 

III.2.b.  Given  a  list  of  statements, 
identify  the  type  of  inspection  system 
described  in  each  statement. 

(2/0) 

(.5/0) 
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COURSE  ITEM 


III.2.C.  Given  a  scenario  describing  a 
maintenance  action,  complete  an 
example  of  an  AFTO  Form  781A. 


III.2.d.  Given  applicable  TOs  and  a 
scenario,  complete  examples  of  AFTO 
Forms  349  and  350. 


ni.2.e.  Given  a  list  of  statements, 
identify  the  statements  that  pertain  to 
Core  Automated  Maintenance  System 
(CAMS). 


III.2.f.  Given  a  list  of  statements, 
identify  the  statements  that  pertain  to 
the  deficiency  reporting  system. 


in.3.  Corrosion  Control 


in.3.a.  Given  a  list  of  statements, 
identify  the  statements  that  pertain  to 
corrosion  control. 


in.4.  Protection,  Safety  Wiring, 
Torque  Indicating  Devices,  and 
CTK  Procedures 


in.4.a.  Given  a  list  of  devices, 
identify  the  device(s)  used  to  protect 
exposed  electrical  coimectors,  open 
pressure  lines,  and  open  waveguides. 


III.4.b.  Given  statements,  select  the 
statements  that  pertain  to  steps 
involved  in  safety  wiring. 


III.4.C.  Given  statements,  select  the 
statements  that  pertain  to  steps 
involved  in  using  torque  indicating 
devices. 


III.4.d.  Given  statements,  select  the 
statements  that  pertain  to  steps 
involved  in  following  CTK 
procedures. 


in.5.  Chafing 


lll.S.a.  Given  a  list  of  statements, 
identify  the  statements  that  define 
chafing. 


#4  MEAN  STD 
HRS  HRS  DEV 


COURSE  ITEM 

POI 

HRS 

#1 

HRS 

n 

HRS 

#3 

HRS 

#4 

HRS 

MEAN 

HRS 

STD 

DEV 

IIL5.b.  Given  a  list  of  statements, 
identify  the  statements  that  describe 
the  causes  of  chafing. 

(2/1) 

in.6.  Cable  Inspection 

3/2 

0/0 

.45/.30 

0/0 

.9/0 

.34/.08 

.37/.  1 

3  ■ 

IIL6.a.  Given  a  list  of  statements, 
identify  the  statements  that  describe 
the  signs  of  deterioration  in  RF 
cables. 

(.45/ 

.30) 

IV.  COMM/NAVAND 
PENETRATION  AIDS  SYSTEM  I 

58.5/ 

20 

11.125/ 

4.5 

47.25/ 

17 

34/ 

12.75 

26.25/ 

7.5 

29.75/ 

10.44 

13.20/ 

4.80 

IV.  1.  Aircraft  Familiarization 

8/2 

0/0 

6.5/1. 5 

6.5/1 .5 

0/0 

3.25/.7 

5 

3.25/ 

.75 

IV.  La.  Given  applicable  TO  and  a 
list  of  major  components,  identify  the 
major  structural  area  where  each 
component  is  located. 

(3/0) 

(3/0) 

(3/0) 

IV.l.b.  Given  applicable  TO  and  a 
list  of  major  components,  identify  the 
major  aircraft  system  described  in 
each  statement. 

(2/1) 

(2/1) 

(2/1) 

rv.  1  .c.  Given  applicable  TO  and  a 
list  of  major  components,  identify  the 
danger  area  described  in  each 
statement. 

(3/1) 

(1.5/.5 

) 

(1.5/ 

.5) 

i 

IV.2.  Aircraft  Avionics 
Familiarization 

3/1 

0/0 

1/1 

3/1 

0/0 

1/.5 

1.22/ 

.5 

IV.2. a.  Given  a  list  of  systems, 
identify  the  systems  which  make  up 
the  A  shred. 

(1/0) 

(1/0) 

IV.2.b.  Given  a  list  of  systems, 
identify  the  systems  which  make  up 
the  B  shred. 

(1/0) 

(1/0) 

IV.2.C.  Given  a  list  of  systems, 
identify  the  systems  which  make  up 
the  C  shred. 

(1/1) 

(1/1) 

(1/1) 

rV.3.  UHF  Communication  and 
Audio  Signal  System 

28/9 

9.5 

/3.5 

24.5/ 

7.75 

24.5/ 

10.25 

14/4.5 

18.1/6. 

5 

6.57/ 

2.68 
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COURSE  ITEM 


#1  #2 
HRS  HRS 


1  . ■■■■,■ .  .  -J 

IV.3.a.  Given  applicable  TO  and  a 
list  of  statements,  identify  the  UHF  or 
audio  signal  system  Line  Replaceable 
Unit  (LRU)  described  in  each 
statement. 

(6/3) 

IV.3.b.  Given  applicable  TO  and  a 
list  of  statements,  identify  the 
statements  that  pertain  to  UHF  or 
audio  signal  theory  of  operation. 

(8/2) 

IV.3.C.  Given  a  list  of  conditions 
existing  in  the  UHF  or  audio  signal 
system  and  applicable  TOs,  trace  the 
signal/data  flow  within  the  UHF  or 
audio  signal  system. 

(8/2) 

IV.3.d.  Given  scenarios  involving 

UHF  or  audio  signal  system 
malfunctions  and  applicable  TOs, 
isolate  the  cause  of  the  malfunctions  in 
each  scenario. 

(6/2) 

IV.4.  Automatic  Direction  Finder 
(ADF)  System 

7/2 

IV.4. a.  Given  applicable  TO  and  a 
list  of  statements,  identify  the 
statements  that  pertain  to  Automatic 
Direction  Finder  (ADF)  system  theory 
of  operation. 

(3.5/1) 

IV.4.b.  Given  a  list  of  conditions 
existing  in  the  ADF  system  and 
applicable  TOs,  trace  the  signal/data 
flow  within  the  ADF  system. 

(3.5/1) 

IV.5.  Crypto  Equipment,  Secure 
Voice 

12.5/6 

IV.5. a.  Given  applicable  TO  and  a 
list  of  statements,  identify  the 
statements  that  pertain  to  secure  voice 
theory  of  operation. 

(6/2) 

IV.S.b.  Given  a  list  of  conditions 
existing  in  the  secure  voice  system 
and  applicable  TOs,  trace  signal/data 
flow  within  the  secure  voice  system. 

(6.5/4) 

V.  COMM/NAVAND 
PENETRATION  AIDS  SYSTEM  H 

34.5/ 

12 

#4  MEAN  STD 
HRS  HRS  DEV 


(3/1.5)  (4.5/ 

2.25) 


(0/0)  (6/1.5) 


(2/.5)  (8/2) 


(1.625/  (6.5/4) 

1) 


6.75/  17.25/ 
2.75  6.75 


COURSE  ITEM 


I  #1  #2  #3  #4  MEAN  STD 

S  HRS  HRS  HRS  HRS  HRS  DEV 


V.l.  Instrument  Landing  System 
(ILS) 

6/1 

V.l.a.  Given  applicable  TO  and  a  list 
of  statements,  identify  the  statements 
that  pertain  to  the  Instrument  Landing 
System  (ILS)  theory  of  operation. 

(4/0) 

V.  1  .b.  Given  a  list  of  conditions 
existing  in  the  ILS  and  applicable 

TOs,  trace  signal/data  flow  within  the 
ILS. 

(2/1) 

V.2.  Tactical  Air  Navigation 
(TACAN)  System 

8/3 

V.2.a.  Given  applicable  TO  and  a  list 
of  statements,  identify  the  statements 
that  pertain  to  Tactical  Air  Navigation 
(TACAN)  System  theory  of  operation. 

(4/1) 

V.2,b.  Given  a  list  of  conditions 
existing  in  the  TACAN  system  and 
applicable  TOs,  trace  signal/data  flow 
within  the  TACAN  system. 

(2/1) 

V.2.C.  Given  scenarios  involving 
TACAN  system  malfunctions  and 
applicable  TOs,  isolate  the  cause  of 
the  malfunction  in  each  scenario. 

(2/1) 

V.3.  Identification  Friend  or  Foe 
(IFF)  System 

6/2 

V,3.a.  Given  applicable  TO  and  a  list 
of  statements,  identify  the  statements 
that  pertain  to  Identification  Friend  or 
Foe  (IFF)  System  theory  of  operation. 

(3.5/1) 

V.3.b.  Given  a  list  of  conditions 
existing  in  the  IFF  system  and 
applicable  TOs,  trace  signal/data  flow 
within  the  IFF  system. 

(2.5/1) 

V.4.  Air-to-air  IFF  Interrogator 
(AAI)  System 

9/3 

V.4.a.  Given  applicable  TO  and  a  list 
of  statements,  identify  the  Air-to-Air 
IFF  Interrogator  (AAI)  System  LRU 
described  in  each  statement. 

(2/0) 

POI 

HRS 

#1 

HRS 

#2 

HRS 

(2/1) 

(0/0) 

(1.5/ 

.75) 

(2/1) 

(.5/.25 

) 

(2/1) 

(3/1) 

(2.25/ 

.75) 

(3/1) 

5.5/3 

2.375/ 

1.5 

4.625/ 

2.25 

(1/1) 

(.5/.5) 

(.5/.5) 

(1/0) 

(0/0) 

(1/0) 

(1.5/1) 

(.375/ 

.25) 

(1.125/ 

.75) 

(2/1) 

(1.5/ 

.75) 

(2/1) 

22.5/8 

0/0 

0 

13/4 

0/0 

0/0 

COURSE  ITEM 


MEAN  STD 


V.4.b.  Given  applicable  TO  and  a  list 
of  statements,  identify  the  statements 
that  pertain  to  AAI  system  theory  of 
operation. 


V.4.C.  Given  a  list  of  conditions 
existing  in  the  AAI  system  and 
applicable  TOs,  trace  signal/data  flow 
within  the  AAI  system. 


V.4.d.  Given  scenarios  involving 
AAI  system  malfunctions  and 
applicable  TOs,  trace  signal/data  flow 
within  the  AAI  system. 


V.5.  Crypto  Equipment,  Mode  4 


V.5.a.  Given  applicable  TO  and  a  list 
of  statements,  identify  the  Mode  4 
system  LRU  described  in  each 
statement. 


V.5.b.  Given  applicable  TO  and  a  list 
of  statements,  identify  the  statements 
that  pertain  to  Mode  4  system  theory 
of  operation.  — 


V.5.C.  Given  a  list  of  conditions 
existing  in  the  Mode  4  system  and 
applicable  TOs,  trace  signal/data  flow 
within  the  Mode  4  system. 


V.5.d.  Given  scenarios  involving 
Mode  4  system  malfunctions  and 
applicable  TOs,  isolate  the  cause  of 
the  malfunction  in  each  scenario. 


VI.  TACTICAL  ELECTRONIC 
WARFARE  SYSTEM 


VI.  1.  Radar  Warning  Receiver 
(RWR)  _ 


VI.  1  .a.  Given  applicable  TO  and  a 
list  of  statements,  identify  the 
statements  that  pertain  to  Radar 
Warning  Receiver  (RWR)  system 
theory  of  operation. 


VI.  1  .b.  Given  a  list  of  conditions 
existing  in  the  RWR  and  applicable 
TOs,  trace  signal/data  flow  within  the 
RWR. 


F-10 


COURSE  ITEM 

POI 

HRS 

#1 

HRS 

#2 

HRS 

#3 

HRS 

u 

HRS 

MEAN 

HRS 

STD 

DEV 

VLl.c.  Given  scenarios  involving 
RWR  system  malfunctions  and 
applicable  TOs,  isolate  the  cause  of 
the  malfunction  in  each  scenario. 

(4/2) 

VI.2.  Internal  Countermeasure 
System  (ICMS) 

9.5/4 

0/0 

0/0 

0/0 

0/0 

VL2.a.  Given  applicable  TO  and  a 
list  of  statements,  identify  the 
statements  that  pertain  to  Internal 
Coimtermeasure  System  (ICMS) 
system  theory  of  operation. 

(4.5/1) 

VI.2.b.  Given  a  list  of  conditions 
existing  in  the  ICMS  and  applicable 
TOs,  trace  signal/data  flow  within  the 
ICMS. 

(5/3) 

Vn.  PENETRATION  AIDS 
SYSTEM 

26/8 

0/0 

0 

0 

4/1 

1.0 

1.73 

Vn.l.  Course  Critique 

1/0 

0/0 

0/0 

0/0 

0/0 

Vn.2.  External  Countermeasures 
System  (PODS) 

6/2 

0/0 

0/0 

0/0 

0/0 

VII.2.a.  Given  applicable  TO  and  a 
list  of  statements,  identify  the 
statements  that  pertain  to  the  External 
Countermeasures  System  (PODS) 
theory  of  operation. 

(4/1) 

VIL2.b.  Given  a  list  of  conditions 
existing  in  the  PODS  and  applicable 
TOs,  trace  signal/data  flow  within  the 
PODS. 

(2/1) 

Vn.3.  Interference  Blanker  System 
(IBS) 

8/2 

0/0 

0/0 

0/0 

4/1 

1.0/.25 

1.73/ 

.43 

Vll.S.a.  Given  applicable  TO  and  a 
list  of  statements,  identify  the 
statements  that  pertain  to  the 
Interference  Blanker  System  (IBS) 
theory  of  operation 

(5/1) 

Vll.S.b.  Given  a  list  of  conditions 
existing  in  the  IBS  and  applicable 

TOs,  trace  signai/data  flow  within  the 
IBS. 

(3/1) 

Vn.4.  Countermeasure  Dispenser 
System 

11/4 

0/0 

0/0 

0/0 

0/0 

F-11 


COURSE  ITEM 

B 

#1 

HRS 

#2 

HRS 

#3 

HRS 

#4 

HRS 

MEAN 

HRS 

STD 

DEV 

vn.4.a.  Given  applicable  TO  and  a 
list  of  statements,  identify  the 
statements  that  pertain  to  the 
Countermeasure  Dispenser  System 
(CDS)  theory  of  operation. 

TOTAL  HOURS 

200.5/ 

68 

17.88/ 

7.25 

69.15/ 

25.7 

53.6/ 

23.0 

57.7/ 

17 

49.6/ 

18.2 

19.2/ 

7.1 

F.3.  Program  of  Instruction,  FID  Course  for  AFS  452X1C 

POI  J4ABF45231C-002,  F-15  AVIONIC  SYSTEMS  SPECIALTY  (COMM,  NAV  AND  PEN 
AIDS),(SHEPPARD  TECHNICAL  TRAINING  CENTER,  Dated  1  June  1989)  is  presented  below  with  the 
following  additions.  POI  HRS  contains  the  training  hours  scheduled  for  each  objective.  Colunms  #1  HRS,  #2  HRS, 
#3  HRS,  and  #4  HRS  contain  estimates  by  4  SMEs  of  the  cross  training  required  for  maintainers  from  another 
aircraft  to  learn  to  maintain  F-15  UHF,  IFF,  and  related  systems.  The  means  and  standard  deviations  of  these 
estimates  are  contained  in  the  final  two  columns. 

Hours  associated  with  individtial  line  items  of  the  POI  are  listed  in  parentheses.  These  hours  are  totaled 
and  displayed  adjacent  to  POI  subheadings.  The  hours  associated  with  the  subheadings  are  totaled  and  displayed 
in  bold  print  adjacent  to  the  POI  major  headings. 

SUMMARY:  SME  ESTIMATES  FOR  FTD  COURSE,  AFS  452X1C 


COURSE  OBJECTIVE 

POI 

#1 

#2 

#3 

#4 

MEAN 

STD 

HRS 

HRS 

HRS 

HRS 

HRS 

HRS 

DEV 

I. 

AIRCRAFT  GENERAL 

21 

12 

0 

5 

0 

4.25 

4.92 

n. 

COMMUNICATION  AND 
NAVIGATION  SYSTEMS 

50 

37 

8.2+ 

17.15 

22.4 

21.18 

10.45 

m. 

TACTICAL  ELECTRONIC 
WARFARE  SYSTEMS 
(TEWS) 

31 

0 

0 

0 

1 

.25 

.43 

IV. 

WIRING  REPAIR 

10 

0 

0 

0 

10 

2.5 

4.3 

TOTAL  HOURS 

112 

49 

8.2+ 

22.15 

33.4 

28.2 

15.0 

F-13 


DETAIL:  SME  ESTIMATES  FOR  FTD  COURSE,  AFS  452X1C 


COURSE  OBJECTIVE 

POI 

HRS 

#1 

HRS 

#2 

HRS 

#3 

HRS 

#4 

HRS 

MEAN 

HRS 

STD 

DEV 

I.  AIRCRAFT  GENERAL 

21 

12 

0 

5 

0 

4.25 

4.92 

1.1.  Course  Orientation 

1 

0 

0 

0 

0 

- 

- 

1.2.  Aircraft  General 

20 

12 

0 

5 

0 

4.25 

4.92 

L2.a.  Given  TOs  and  an  aircraft, 
perform  an  aircraft  safe  for 
maintenanceinspection  with  no 
instructor  assistance. 

(6) 

(6) 

(0) 

(1.5) 

(0) 

(1.88) 

(2.46) 

I.2.b.  Given  TOs,  an  aircraft, 
external  power  unit,  a  ground  air 
conditioning  unit,  and  working  as  a 
team  member,  perform  external  power 
and  cooling  air  application  and 
removal  procedures  with  no  more  than 
three  instructor  assists  per  team. 

(6) 

(6) 

(0) 

(1.5) 

(0) 

(1.88) 

(2.46) 

L2.C.  Given  TOs,  an  aircraft, 
external  power  unit,  hydraulic  power 
unit,  and  working  as  a  team  member, 
perform  external  utility  hydraulic 
power  application  and  removal 
procedures  with  no  more  than  four 
instructor  assists  per  team. 

(8) 

(0) 

(0) 

(2) 

(0) 

(.5) 

(.866) 

n.  COMMUNICATION  AND 
NAVIGATION  SYSTEMS 

50 

37 

8.2+^ 

17.15 

22.4 

21.18 

10.45 

n.l.  Organizational  Maintenance 
Systems  of  the  F-15  Communication 
System 

12 

12 

4 

2.15 

5.5 

5.91 

3.71 

11. 1. a.  Given  TOs,  F-15 

Aircraft/Flight  simulator/TFE-15,  and 
AGE,  perform  an  Aux  Receiver  and 
ICCP  BIT  Checkout  with  no  more 
than  two  instructor  assists. 

(3) 

(3) 

(.9) 

(.9) 

(1.5) 

(1.585) 

(.858) 

Il.l.b.  Given  TOs,  F-15 
Aircraft/TFE-15,  AGE,  and  working 
as  a  team  member,  perform  a  UHF 
Communication  and  Audio  Signal 
System  Operational  checkout  with  no 
more  than  five  instructor  assists  per 
team. 

(5) 

(5) 

(1.5) 

(1.25) 

(2.0) 

(2.170) 

(1.308) 

^  The  plus  sign  (+)  indicates  that  the  SME  believed  a  training  requirement  existed  but  that  the  SME  felt 
unqualified  to  make  an  estimate. 
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COURSE  OBJECTIVE 

POI 

HRS 

#1 

HRS 

#2 

HRS 

#3 

HRS 

#4 

HRS 

MEAN 

HRS 

STD 

DEV 

ILLc.  Given  TOs,  F-15 

Aircraft/Flight  simulator/TFE-15,  and 
AGE,  perform  an  ADF  BIT  checkout 
with  no  more  than  two  instructor 
assists. 

(2) 

(2) 

(.6) 

(0) 

(1.0) 

(.9) 

n.l.d.  Given  TOs,  F-15 
Aircraft/TFE-lS,  and  AGE,  perform 
an  ADF  operational  checkout  with  no 
more  than  two  instructor  assists. 

(2) 

(2) 

(1) 

(0) 

(1.0) 

(1.0) 

(.707) 

n.2.  Organizational  Maintenance 
of  the  F-15  Navigation  System 

38 

25 

4.20-1- 

15 

16.9 

15.3 

7.4 

II.2.a.  Given  TOs,  F-15 

Aircrafl/TFE,  AGE,  Test  Equipment, 
and  working  as  a  team  member, 
perform  an  ILS  Operational  Checkout 
with  no  more  than  three  instructor 
assists  per  team. 

(3) 

(0) 

(0) 

(0) 

(0) 

n.2.b.  Given  TOs,  F-15 

Aircraft/Flight  Simulator/TFE-15,  and 
AGE,  perform  a  TACAN  BIT 
checkout  with  no  more  than  two 
instructor  assists. 

(3) 

(0) 

(0) 

(0) 

(0) 

IL2.C.  Given  TOs,  F-15 
Aircraft/TFE-15,  and  AGE,  perform 
a  TACAN  operational  checkout  with 
no  more  than  three  instructor  assists. 

(3) 

(0) 

(0) 

(0) 

(0) 

II.2.d.  Given  TOs,  F-15  Aircraft, 

AGE,  test  equipment,  and  working  as 
a  team  member,  perform  fault 
isolation  procedures  of  Tactical  Air 
Navigation  System  with  no  more  than 
five  instructor  assists  per  team. 

(4) 

(0) 

(0) 

(0) 

(0) 

IL2.e.  Given  TOs,  AFTO  Forms  349 
and  350,  and  simulated  maintenance 
data,  document  on-equipment 
maintenance  with  no  more  than  three 
errors  per  AFTO  Form. 

(3) 

(3) 

(0) 

(0) 

(0) 

II.2.f.  Given  TOs,  F-15 

Aircraft/Flight  Simulator/TFE-15,  and 
AGE,  perform  an  IFF  Transponder 

BIT  Checkout  with  no  more  than  two 
instructor  assists. 

(3) 

(3) 

(.9) 

(3) 

(.9) 

(1.95) 

(1.05) 

F-15 


COURSE  OBJECTIVE 

POI 

HRS 

#1 

HRS 

#2 

HRS 

#3 

HRS 

#4 

HRS 

MEAN 

HRS 

STD 

DEV 

il.2.g.  Given  TOs,  F-15  Aircraft, 

AGE,  test  equipment  and  working  as 
a  team  member,  perform  an  IFF 
operational  checkout  using  AN/APM- 
424  with  no  more  than  three  instructor 
assists  per  team. 

(3) 

(3) 

(1.5) 

(3) 

(3) 

(2.625) 

(.6495) 

II.2.h.  Given  TOs,  F-15 
Aircraft/TFE-15,  and  AGE,  perform 
an  Air-to-Air  and  Mode  4  IFF  BIT 
checkout  with  no  more  than  two 
instructor  assists. 

(6) 

(6) 

(1.8) 

(3) 

(3) 

(3.45) 

(1.552) 

Given  TOs,  F-15 
Aircraft/TFE-15,  AGE  test 
equipment,  and  working  as  a  team 
member,  perform  an  Air-to-Air  IFF 
operational  checkout  using  AN/APM- 
349  with  no  more  than  four  instructor 
assists  per  team. 

(6) 

(6) 

+ 

(3) 

(6) 

(5) 

(1.41) 

II.2.j.  Given  TOs,  F-15  Aircraft, 

AGE,  test  equipment,  and  working  as 
a  team  member,  perform  fault 
isolation  procedures  of  the  Air-to-Air 
IFF  system  with  no  more  than  five 
instructor  assists  per  team. 

(4) 

(4) 

(0) 

(3) 

(4) 

(2.75) 

(1.64) 

in.  TACTICAL  ELECTRONIC 
WARFARE  SYSTEMS  (TEWS) 

31 

0 

0 

0 

1 

.25 

.43 

m.l.  Organizational  Maintenance 
of  the  Tactical  Electronic  Warfare 
Systems  (TEWS) 

31 

0 

0 

0 

1 

.25 

.43 

Ill.l.a.  Given  TOs,  F-15 

Aircraft/Flight  Simulator/TFE-15,  and 
AGE,  perform  a  RWR  BIT  with  no 
more  than  two  instructor  assists. 

(3) 

(0) 

(0) 

(0) 

(0) 

(0) 

(0) 

Ill.l.b.  Given  TOs,  F-15  Aircraft 
TFE-15  AN/APM-427  Test  Set, 

AGE,  and  working  as  a  team 
member,  perform  an  Improved  Radar 
Simulator  Checkout  with  no  more  • 
than  three  instructor  assists  per  team. 

(3) 

(0) 

(0) 

(0) 

(0) 

(0) 

(0) 

(O)IILl.c.  Given  TOs,  identify 
relationship  of  basic  facts  and 
principles  about  Electronic  Warfare 
Warning  System  (EWWS)  with  a 
minimum  of  70  percent  accuracy. 

(6) 

(0) 

(0) 

(0) 

(0) 

(0) 

(0) 

F-16 


COURSE  OBJECTIVE 

POI 

HRS 

#1 

HRS 

#2 

HRS 

#3 

HRS 

#4 

HRS 

MEAN 

HRS 

STD 

DEV 

ni.  1  .d.  Given  TOs,  identify 
malfunctions  for  the  EWWS  with  a 
minimum  of  70  percent  accuracy. 

(6) 

(0) 

(0) 

(0) 

(0) 

(0) 

(0) 

ni.  1  .e.  Given  TOs,  F-15 
Aircraft/Flight  Simulator/TFE-15,  and 
AGE,  perform  an  EWWS  BIT  with 
no  more  than  three  instructor  assists. 

(3) 

(0) 

(0) 

(0) 

(0) 

(0) 

(0) 

Ill.l.f.  Given  TOs,  F-15  Aircraft, 
AN/ALM-231  Test  Set,  AGE,  and 
working  as  a  team  member,  perform  a 
Walk-Around  Transportable  Test  Set 
procedure  with  no  more  than  three 
instructor  assists  per  team. 

(3) 

(0) 

(0) 

(0) 

(0) 

(0) 

(0) 

m.l.g.  Given  TOs,  F-15 
Aircraft/Flight  Simulator/TFE-15, 

AGE,  and  test  equipment,  perform  an 
ICMS  BIT  Checkout  with  no  more 
than  two  instructor  assists. 

(2) 

(0) 

(0) 

(0) 

(0) 

(0) 

(0) 

Ill.l.h.  Given  TOs,  F-15  Aircraft, 
AGE,  and  test  equipment,  perform  an 
ICMS  pressurization  checkout  with  no 
more  than  two  instructor  assists. 

(3) 

(0) 

(0) 

(0) 

(0) 

(0) 

(0) 

ni.l.i.  Given  TOs,  F-15 

Aircraft/Flight  Simulator/TFE-15,  and 
AGE,  perform  an  IBS  BIT  with  no 
more  than  two  instructor  assists. 

(2) 

(0) 

(0) 

(0) 

(1) 

(.25) 

(.433) 

IV.  WIRING  REPAIR 

10 

0 

0 

imon 

10 

2.5 

4.3 

IV,  1.  Wiring  Repair 

10 

0 

0 

0 

10 

2.5 

4.3 

IV.  1. a.  Given  TOs,  aircraft  ID 
number,  aircraft  model,  reference 
designator  number,  and  pin  number, 
locate  information  and  tools  to  repair 
a  defective  wire  with  no  more  than 
four  errors. 

(5) 

(0) 

(0) 

(0) 

(5) 

(1.25) 

(2.17) 

IV.  1  .b.  Given  TO,  TDR,  and  a 
sample  test  cable. 

(2) 

(0) 

(0) 

(0) 

(2) 

(.5) 

(.87) 

IV.l.c.  Given  applicable  TOs,  a 
solderless  connector  repair  kit,  and  a 
multi-pin  connector  with  wire  and 
pins:  remove,  reterminate,  reinsert  a 
pin  with  no  more  than  two  instructor 
assists. 

(3) 

(0) 

(0) 

(0) 

(3) 

(.75) 

(1.30) 

TOTAL  HOURS 

112 

49 

8.2-1- 

22.15 

33.4 

28.2 

15.0 

F-17 


F.4.  Specialty  Training  Standard  (STS)  452X1  (Dated  February  1991.) 

The  following  table  is  derived  from  STS  452X1 ,  the  standard  which  describes  the  "common  tasks, 
knowledge,  and  technical  references  (TR)  necessary  for  airmen  to  perform  duties  at  the  3-,  5-  and  7-skill 
level  AFSC  in  the  Integrated  Organizational  Avionics  F-15  Systems  ladder  of  the  Airman  Avionics 
System  Career  Field. "  The  complete  STS  was  presented  to  three  SMEs  who  reviewed  it  to  identify  the 
line  items  necessary  for  the  cross  training  of  an  ATF  avionics  maintainer  to  F-15  UHF  and  IFF.  While 
the  SMEs  reviewed  the  entire  document,  only  those  portions  identified  by  at  least  one  SME  as  necessary 
for  cross  training  are  displayed  below.  #3  TIME  contains  the  joint  response  of  SME  respondents  2  and 
3,  and  #4  TIME  contains  the  response  of  the  last  SME;  the  final  column  contains  the  mean  of  these 
estimates. 


TASK  TITLE 

#3  TIME 
(HRS) 

#4  TIME 
(HRS) 

MEAN 

TIME 

15; 

USE  TEST  EQUIPMENT 

5.5 

2,75 

15a. 

Boresight 

15b. 

RF  tester 

15c. 

WOW/proximity  box 

15d. 

Waveguide  pressure  tester 

15a. 

Memory /loader  verifier 

15f. 

Transmission  tester 

15g. 

Fuel  quantity  tester 

15h. 

Hydrometer 

15i. 

Compass  calibrator 

15j. 

Standby  compass  calibrator 

15k. 

TTU-205  C/E 

151- 

Automatic  Flight  Control  test  set 

15m. 

Linear  gauge  (Pogo  Stick) 

15n. 

Thru-line  WATT  meter 

15o. 

ILS  test  set 

15p. 

IFF  test  set 

1 

.5 

15q. 

AAI  test  set 

4.5 

2.25 

15r. 

Radar  target  simulator 

15s. 

HPRF  target  generator 

15t. 

Time  domain  reflectometer 

15u. 

Antenna  diode  tester 
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TASK  TITLE 


UIF  COMM^NiCATIOH  AND  AUDIO 
SIGNAL  SYSTEM  _______ 


Theory  of  operation 

Trace  wiring,  system,  and  interface  diagrams 
Perform  operational  checkout  and  BIT 
Isolate  malfunctions 
Remove  system  LRU(s) 

Install  system  LRU(s) 

Code/Decode  KY  unit 


AUTOMATIC  DIRECTION  FINDER  (ADF) 

SYSTEM  _ 


Theory  of  Operation 

Trace  wiring,  system,  and  interface  diagrams 
Perform  operational  checkout  and  BIT 
Isolate  malftinctions 
Remove  system  LRU(s) 

Install  system  LRU(s) 


IDENTIHCATION  FRIEND  OR  FOE  ^FF) 

SYSTEM  AND  MODE  4  SYSTEM 


Theory  of  operation 

Trace  wiring,  system,  and  interface  diagrams 
Perform  operational  checkout  and  BIT 
Isolate  malfunctions 
Remove  system  LRU(s) 

Install  system  LRU(s) 

Code/decode  Mode  4 


AIR-TO-AIR  IFF  INTERROGATOR  (AAl) 

SYSTEM  AND  MODE  4  SYSTEM _ 


Theory  of  operation 

Trace  wiring,  system,  and  interface  diagrams 
Perform  operational  checkout  and  BIT 
Isolate  malfunctions 
Remove  system  LRU(s) 

Install  system  LRU(s) 

Code/decode  Mode  4 


#3  TIME  H  TIME  MEAN 
(HRS)  (HRS)  TIME 
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F.5.  OJT  Questionnaires 


Questionnaires  were  presented  to  new  5-skill  level  airmen  and  OJT  instructors  to  obtain  their  estimates 
of  the  OJT  training  requirement  for  avionics  maintenance  personnel  on  the  UHF  and  IFF  systems  of  the  F-15. 
The  survey  responses  are  displayed  and  tabulated  below;  the  questionnaires  themselves  follow. 

Each  pair  of  columns  corresponds  to  the  respondent  with  the  same  number  in  the  preceding  survey 
response  tables.  Columns  2A,  3A,  and  4A  are  the  responses  as  they  were  supplied  by  the  respondents; 
columns  2B,  3B,  and  4B  are  those  responses  converted  to  OJT  time  based  on  48  work  weeks  per  year,  5 
work  days  per  week,  and  8  work  hours  per  day.  Colunm  3A  contains  data  from  the  only  survey  respondent 
who  is  currently  an  OJT  instructor.  Based  upon  the  instructor’s  statement  that  the  3  to  5  skill  level  OJT 
period  typically  lasts  twelve  months,  the  other  survey  responses  were  standardized  to  twelve  month  training 
periods  as  well. 

Using  the  survey  responses  as  standardized  to  twelve  month  OJT  periods,  UHF  training  requires  57.0 
hours  (std=.9  hrs)  and  IFF  training  requires  145.9  hours  (std=25.9hrs). 


SURVEY  QUESTION 

#2A 

#2B 

#3A 

#3B 

#4A 

#4B 

1.  A.  Initial  skills 

course 

9/17/90- 

1/29/91 

5  months 

- 

- 

6  months 

6  months 

B.  FTD 

2/1/91- 

3/1/91 

1 

- 

- 

3 

3 

C.  OJT 

3/1/91- 

3/31/92 

13 

12  months 

12  months 

5/89-7/91“ 

27 

2.  %  OJT 

30% 

624  hrs 

30% 

576  hrs 

30% 

1296  hrs 

3.  %  UHF 

10% 

62.4  hrs 

10% 

57.6  hrs 

10% 

129.6  hrs 

4.  63A 

8.0% 

49.9  hrs 

80% 

46.1  hrs 

40% 

51.8  hrs 

63B 

1.0% 

6.2 

10% 

5.8 

10% 

13.0 

63C 

.2% 

1.2 

5% 

2.9 

10% 

13.0 

63D 

.5% 

3.1 

5% 

2.9 

10% 

13.0 

TotalUHF 

60.4  hrs 

57.7  hrs 

129.6  hrs 

5.  %IFF 

30% 

187.4  hrs 

25% 

144.0  hrs 

20% 

259.2  hrs 

6.  65A 

15% 

93.6  hrs 

45% 

64.8  hrs 

45% 

116.6  hrs 

65B 

15% 

93.6 

45% 

64,8 

45% 

116.6 

65C 

1% 

6.2 

10% 

14.4 

10% 

25.9 

Total  IFF 

193.4  hrs 

144.0  hrs 

259  hrs 

Standardized  to  12 
months: 

UHF 

55.8  hrs 

57.7  hrs 

57.6  hrs 

IFF 

178.5 

144.0 

115.1 

TOTAL  HOURS 

234.3  hrs 

201.7  hrs 

172.7  hrs 

Includes  time  deployed  to  Operation  Desert  Shield/Storm. 
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OJT  Questionaire  for  New  5-levels 
Percent  Time 

1.  Please  list  the  training  courses  you  attended,  including  dates,  from  completion  of  BMT  until  you 
were  awarded  your  5-level  (45251C). 

Example: 

Initial  skill  course:  Oct  1990  -  Feb  1991 

FTD  course:  Mar  1990  -  Mar  1990 

OJT  (upgrade  training)  at  Langley  Apr  1991  -  Mar  1992 

2.  On  average,  what  percentage  each  day  did  you  spend  in  OJT? 

3.  What  percentage  of  your  OJT  time  was  spent  on  the  UHF  communication  system? 

4.  What  percentage  of  your  UHF  OJT  time  was  spent  on  each  3-digit  WUC  set  comprising  the  UHF 
communication  system  (see  below)? 


UHF  Communications 


WUC 

63A 

63B 

63C 

63D 


System 

UHF  Communications 
UHF  Communications 
UHF  Communications 
UHF  Communications 


Subsystem 
UHF  Comm  set 
Integ  CNI  control  set 
Intercomm  system 
SEEK  TALK 


5.  What  percentage  of  your  OJT  time  was  spent  on  the  IFF  system? 

6.  What  percentage  of  your  IFF  OJT  time  was  spent  on  each  3-digit  WUC  set  comprising  the  IFF 
system  (see  below)? 

Identification  Friend  or  Foe 


WUC 

System 

Subsystem 

65A 

IFF 

,  Transponder  set 

65B 

IFF 

IFF  interrogator  set 

65C 

IFF 

IFF  antenna  set  switch 

Figure  F.l:  OJT  Questionnaire  for  New  5-LeveIs 
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OJT  Questionaire  for  OJT  Instructors 
Percent  Time 


1 .  On  average,  what  percentage  each  day  does  a  3-level  spend  in  OJT  (upgrade  training)? 

2.  On  average,  how  many  months  does  a  3-level  spend  in  OJT  before  upgrade  to  the  5-level? 

3.  What  is  the  average  ratio  of  students  to  instructors  during  the  time  a  3-level  spends  in  OJT? 

3.  On  average,  what  percentage  of  a  3-levers  OJT  time  is  spent  on  the  UHF  communication  system? 

4.  On  average,  what  percentage  of  a  3-levers  UHF  OJT  time  is  spent  on  each  3-digit  WUC  set 
comprising  the  UHF  communication  system  (see  below)? 

UHF  Communications 

WUC  System  Subsystem 

63A  UHF  Communications  UHF  Comm  set 

63B  UHF  Communications  Integ  CNI  control  set 

63C  UHF  Communications  Intercomm  system 

63D  UHF  Communications  SEEK  TALK 

5.  On  average,  what  percentage  of  a  3-lever s  OJT  time  is  spent  on  the  IFF  system? 

6.  On  average,  what  percentage  of  a  3-levers  IFF  OJT  time  is  spent  on  each  3-digit  WUC  set 
comprising  the  IFF  communication  system  (see  below)? 

Identification  Friend  or  Foe 
WUC  System  Subsystem 

65A  IFF  -  Transponder  set 

65B  IFF  IFF  interrogator  set 

65C  IFF  IFF  antenna  set  switch 


Figure  F.2:  OJT  Questionnaire  for  OJT  Instructors 
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